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ABSTRACT 



This handbook is a general guide for starting a workplace literacy program. 
It outlines methods for establishing a program, from the early stages of 
planning through implementation to program evaluation. It discusses some of 
the literacy demands that workers face and how workplace education programs 
can address them. The key elements of planning a workplace literacy program 
include establishing a planning group, analyzing the need for education, 
locating funding sources, and gaining the support of management and 
employees. Developing a program involves conducting a literacy task analysis 
and designing relevant curriculum using materials from learners’ jobs. It dso 
involves recruiting learners to a program. Program evaluation is discussed from 
two perspectives. Formative evaluation examines the program while it is still 
under way so that changes can be made. Sununative evduation determines how 
effectively goals weie met and looks at the impact of the program on learners 
and job productivity. 
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Introduction 



This handbook is intended to serve as a general guide for starting a 
workplace literacy program. It outlines methods for estabUshing a program, 
from the early stages of planning through implementation to program 
evaluation. The definition of literacy used in this guide is a broad one, 
encompassing the basic skills of reading, writing, and mathematics; as they 
relate to the use of print materials such as manuals, forms, and graphs. 
Excluded from the definition is the more technical workplace training involved, 
for example, in learning how to operate new machinery. However, such 
technical training may require prelitninary literacy education for some workers, 
who may have difficulty reading the training materials or related technical 
manuals. Thus the scope of what a workplace literacy educator may do is 
somewhat vague, but its core lies in the area of basic skills. 

Chapter One discusses some of the literacy demands that workers face and 
how workplace education programs can address them. Trends influencing 
programs include increased use of literacy in the workplace, the growing 
service sector including more temporary and part-time jobs, and an increase in 
the number of non-English speaking workers. Effective programs address 
individual learner needs through long-term education and the use of custom- 
designed instruction. 

Chapter Two describes key elements of planning a workplace literacy 
program, including establishing a planning group, analyzing the need for 
education, locating funding sources, and gaining the support of management 
and employees. It is important to address the needs of the various groups within 
a workplace in order to encourage the development of a shared vision. 

Chapter Three explains how to conduct a literacy task analysis and how to 
design relevant curriculum using materials from learners’ jobs. This custom- 
designed curriculum maj supplemented with suitable off-the-shelf materials 
and the development Oi. job aids. This chapter also describes methods for 
recruiting learners to a program. 

Chapter Four discusses evaluating the program from two perspectives. 
Formative evaluation examines the program while it is still under way so that 
changes can be made. Summative evaluation uses learner assessments and other 
measures to determine how effectively goals were met and to look at the impact 
the program may have had on job productivity. 

This guide includes several important fea.tures to assist you as you plan your 
workplace literacy program. For quick reference, summary boxes appear below 
each major heading in the text. These will provide a brief overview of the key 
points to remember. In addition, an appendix c''ntains a set of overheads that 
provide a broad outline of each chapter. These are intended to serve as teaching 
aids so that the guide can be presented as a series of workshops for staff 
development. Besides providing a complete reference list, the bibliography also 
indicates sources for further reading on topics covered in each chapter, such as 
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references that describe in detail how to perform a literacy task analysis or 
conduct formative interviews. 

It is recommended that you read the entire guidebook before 
developing a program or offering any courses. This is because 
some of the steps necessary to establish a new program will 
overlap with others. For example, even though some parts of the 
evaluation are performed after learners have completed a course, 
effective evaluation requires developing assessments that are 
administered to learners before they begin taking courses. The 
guidelines discussed here will assist you in all stages of program 
development, from establishing a planning group to evaluating the 
program’s impact. 



Chapter One 



CURRENT PRACTICES IN WORKPLACE LITERACY 
PROGRAMS 

Over the past two decades, research has identified the challenges that 
workplace literacy programs face. For example, we have learned that 

• there are several different workplace literacy problems calling for 
different solutions, 

• improvement takes a significant amount of learner practice time, 

• transfer of learning to new applications is severely limited, and . 

• significant learning loss occurs within a few weeks if skills are hot 
practiced. 

Recent trends in the labor market and in the population of workers seeking 
employment have also begun to influence workplace education. These trends 
include 

• a shift in employment opportunities from relatively low-skilled jobs in 
manufacturing to higher skilled jobs in the service sector; 

• a restructuring toward more efficient, “high-performance” workplaces 
which require higher skill levels of their workers; 

• a general increase in the use of literacy in workplaces; 

• an increase in temporary and part-time work; 

• a shortfall in the level of skills workers bring to employment; and 

• an increase in the number of workers with limited abilities to use English. 

Workplace literacy programs operating within these constraints are 
characterized by 



• instmction that addresses individual learners’ needs and is long term; 

• active involvement by major stakeholders (e.g., management, employee 
organizations, instructors, and the learners themselves); and 
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• a custom-designed curriculum that integrates basic skills instruction with 
workplace applications and materials. 

Although effective programs incorporate these elements, few have the 
necessary resources to incorporate them all at the outset. Most likely, you will 
have to begin with one class, such as a GED (General Educational 
Development) program or perhaps a short job-related course that teaches some 
basic slalls, and then expand gradually to include more customized instruction. 
Thus, an effective literacy program that meets the needs of all workers can be 
achieved gradually, over a longer period of time. 

DEMANDS AND CONSTRAINTS 



• Multiple program strands should address multiple problems. 

• Improvement takes significant learner practice tLne. 

• Transfer to new applications is severely limited. 

• Significant learning loss occurs without regular practice. 



HI Multiple Program Strands Should Address Multiple Problems 

Because worker skills and needs vary widely, an effective workplace 
program should offer several sorts of instmction. These may include tutoring 
for very low-level literates, GED courses for those without high school 
diplomas, targeted skills for brash-up courses preceding technical training, and 
workplace-based English as a Second Language classes in workplaces with 
many ESL speakers. 

A program with several different sorts of instruction is much more likely to 
succeed. Learners who experience success in one class are more likely to take 
additional classes. This is extremely important because most single workplace 
classes are too brief to have much impact. To make major skills gains, workers 
must continue learning and practicing for hundreds of hours as they move from 
class to class and strand to strand. 

EXAMPLE: In one manufacturing plant, a multi-strand program was 
started with a skills enhancement strand where learners enrolled on a 
voluntary basis and received general instruction in areas where they 
believed they needed the most review. As the program continued, 
additional strands were added to address specific skills related to 
jobs that were performed most frequently around the plant. Later, a 
math skills strand was developed that supported the plant’s new 
quality control program, contributing to over^l productivity. 

As you become more familiar with your organization’s concerns, it is 
possible to incorporate new strands gradually to meet learners’ demands. 

H Improvement Takes Significant Learner Practice Time 

Training material and technical reading materia! in the workplace tend to 
range from upper high school to beginning college difficulty levels (Mikulecky, 
1982; Rush, Moe & Storlie, 1986; Sticht, 1975). Though some learners with 
high school diplomas may only need a short brush-up course (i.e., 30-50 
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hours) to learn to comprehend such materials, other learners may take as much 
as several hundred hours of instraction. 

A child in school takes 180 full days of practice, on average, to improve a 
grade level in reading and mathematical abilities. Adults are more efficient than 
diis, but it still takes an average of 100-120 hours of practice and instruction for 
adults in adult basic education classes to make a grade level improvement. 
Although some individuals may improv.' more rapidly, the best award-winning 
programs still take 50-60 hours to help learners improve a grade level in generd 
literacy skills (Mikulecky, 1989)'. TWs means that a worker who has difficulty 
reading simple sentences will take hundreds of hours of practice and instmction 
before he or she can easily read workplace memos, manuals, and safety 
instructions. 

Since workplaces cannot afford to offer the thousands of hours of 
instruction provided by public schools, it is important to provide workplace 
instruction which multiplies practice outside of classes. This means linking 
instruction to materials and opportunities for practice that exist in the work and 
home lives of learners. Much like a diet, exercise, or health program, an 
effective workplace literacy program must help adults change their lifestyles to 
include more literacy practice. 

EXAMPLE: In an insurance company, a technical manual became the 
basis for instruction in how to scan the table of contents to find 
information about various types of claims, a task that clerical 
personnel performed frequently. Because the classroom exercise 
closely paralleled an everyday job task, workers could practice and 
learn ^s skill more quickly. 

H Transfer to New Applications Is Severely Limited 

Many people mistakenly think that any sort of literacy instruction will 
translate to improved learner skills on the job. College-educated adults often 
make this mistake because they are able to apply their own high level literacy 
skills to learning and meeting the demands of new jobs. They can do this 
because the literdly thousands of hours that they have invested in literacy use 
has provided them with a high level of skills and excess general literacy 
capacity. 

The same situation is not true of low and middle level literates, however. 
They have practiced little and have no excess capacity. In fact, for a person who 
has not read much and stmggles with a newspaper, new workplace literacy 
demands may be more difficult than anything ever read before. They have no 
excess of general literacy capacity to support learning new technical literacy 
vocabulary and strategies. A couple of months of general literacy instruction at 
four hours a week will provide only a slight improvement in general literacy 
skills, and usually shows no noticeable impact on the ability to apply general 
literacy to technical applications. 

In fact, research indicates that there is a severe limitation to how much 
literacy transfers from one type of task to other types of tasks (Kirsch & 
Jungeblut, 1986). The limitations of literacy transfer have serious implications 
for worki)lace literacy programs. This is especially true if programs attempt to 
use traditional, school-type materials. Sticht (1982) found that general literacy 
instruction only transferred minimally to job applications. He now recommends 
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a “functional context” approach, which teaches skills using materials that 
learners use on a daily basis (Sticht, 1982). If a small amount of instruction is 
all that is available and improved workplace practice is the goal, it is wiser to 
target instraction directly upon specific workplace demands. 

EXAMPLE: In the insurance company just mentioned, clerical 
workers were able to decrease the time required to find claims 
information because their instruction used the same manual that they 
used daily on the job. 

HI Significant Learning Loss Occurs Without Regular Practice 

Lack of transfer is related to the problem of learning loss. When a person 
caimot transfer learning to real-world situations, opportunities to practice new 
skills during everyday activities on the job are lost. Sticht (1982) reports that 
military enlisted men improved in general literacy abilities while they were in 
general literacy classes, but that 80% of the gains were lost within eight weeks. 
However, when job-related materials were used to teach literacy abilities, 
learning gains were retained. This is most likely because learners could continue 
practicing on the job the abilities that they had mastered. 

If it is to be effective in the short term, instruction should not overly 
emphasize general materials that have no relationship to materials workers see 
daily. The timing for instmction in workplace skills is also important. Preparing 
learners for the basic skill demands of new jobs may be wasted if they must 
wait several months before they are able to apply and practice their new 
learning. 

EXAMPLE: Since the instruction for insurance clerical personnel 
described above was designed to correspond closely to their jobs, 
learners were able to gain additional practice each time they searched 
for claims information on the job. In addition, timing the instruction 
to correspond closely to the time when skills would be needed also 
enhanced learners’ ability to recall what they had just learned. This 
is sometimes called just-in-time training. 



In summary, when designing a literacy program, it is necessary to consider 
four constraints that will affect the program’s success. Because workers’ skills 
vary widely, you should design a multi-strand program that can address a 
variety of learner needs. Workplace literacy also requires a long-tern 
commitment, since improvement requires a significant amount of practice time 
(e.g., average adult basic education programs take about 100 hours for the 
equivalent of one grade level gain). There is no such thing as a “quick-fix” 
solution to literacy skill problems. To gain improvement in job performance, 
literacy skills should be taught using job-related materials, not general 
educational materials that do not provide a clear link to the job. Finally, 
instruction must provide opportunities for practice in a timely manner, or else 
newly-acquired skills will te lost before they can be used on the job. 

Educators can provide opportunities for additional practice by also 
emphasizing mading materials from home that are of general interest to learners. 
Even when those materials are not similar to those used in the workplace, 
benefits can still be gained when learners spend additional time reading. It may 
also be possible to incorporate documents from home (e.g., tables from 
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newspaper articles, diagrams found in home repair manuals, bills, store ads) 
that parallel job materials. For example, in a course that teaches the use of 
percentages on the job, examples of store ads advertising percentages off 
regular prices could used to provide additional practice at this skill. 



LABOR Market Trends 

There are a number of recent trends in the labor market that have started to 
influence the need for workplace education and the nature of the programs 
provided. 

• The Shift From Manufacturing to the Service Sector 

• The Growth in High-Performance Workplaces 

• Increased Use of Workplace Literacy 

• Increase in Temporary and Part-Time Work 

THE SHIFT FROM MANUFACTURING TO THE SERVICE SECTOR 



• Since 1965, the United States has experienced a large shift in 
employment from manufacturing to service sector jobs. 

• In the past, it was possible to earn more in manufacturing jobs that 
demanded less skill. 

• Many workers find that they must now have higher literacy skills to 
get jobs that pay less. 

• This can have serious motivational consequences for those workers, 
which must be addressed by workplace educators. 



In the last 30 years, the United States has experienced a dramatic shift in 
employment from jobs in the manufacturing industries to jobs in the service 
sector. In 1965, 29.7% of the workforce was employed in manufacturing, but 
by 1993, that number had decreased to only 16.3% of workers. On the other 
hand, the service sector has increased from employing 63.9% of the workforce 
in 1965 to 79.0% in 1993 (Monthly Labor Review, 1966, 1994). In the past, it 
was possible for many people to earn middle-class incomes while worWng in 
manufacturing, and they were not required to have high school diplomas or 
advanced literacy skills to remain employed. However, the shift towards 
employment in services has meant that people must have higher literacy skills to 
get jobs that pay less (Borjas & Ramey, 1994; Gittleman, 1994; Moore & 
Blake, 1992). 

This increase in the number of demanding jobs that pay less may have 
serious motivational consequences for workers. If a program is to be 
successful, the workers’ motivation for enrolling in the program should be 
considered during the early stages of planning. For example, some workers 
may enroll to improve their chances for advancement to higher paying 
positions. They may also fear losing their jobs due to a perceived inability to 
keep up with increased skill demands, or perhaps they may be seeking new 
employment. Some may see it as an opportunity never offered to them before to 
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improve tijeir basic skills and work life. To accomplish its goals, the program 
must have the support of workers by meeting their goals and needs. 

EXAMPLE: In a food manufacturing plant, some production 
^ employees were enrohmg in a program because a new computer 
system was being installed, and they feared that they might not be 
able to adjust to the demands of this new technology. Instruction 
incorporated job simulation exercises that demonstrated how to use 
the new computer system. Therefore, it taught basic technical skills 
while addressing a motivational need. 

THE GROWTH IN HIGH-PERFORMANCE WORKPLACES 



• Increasing numbers of businesses are adopting high performance 
techniques, such as quality assurance teams and greater use of 
technology. 

• High-performance workplaces have greater demand for higher skilled 
workers. 

• Fewer jobs are available but with more responsibility. 

• Education should focus on skills that will help workers cope with 
those additional responsibilities. 



One trend that affects both service sector and manufacturing jobs is the 
extent to which organizations are forming “high performance workplaces.” The 
goal of a high performance workplace is to increase productivity by cutting 
bureaucracy and putting greater responsibility for quality on front-line workers. 
For example, in manufacturing plants that employ high performance practices, 
production em.ployees may be asked to track machine efficiencies on a chart and 
then compute the total production efficiency for their shift, both tasks 
previously performed by supervisors. 

Currently, approximately 35% of businesses are employing some form of 
high performance workplace technique, such as 

• quality assurance teams, 

• increased use of technology, 

• statistical process control, and 

• just-in-time production (i.e., producing parts as needed rather than 

accumulating an inventory). 

These techniques often involve layoffs as work is reorganized. The skill 
demands made on remaining workers are increased as a result. (For examples 
of high performance workplace practices, see Bassi, 1992; Gordon, 1993; 
Osterman, 1993; Richman, 1994). In many workplaces, the employees are not 
able to make the necessary adjustments at first, and extensive workplace literacy 
education is required to improve their skills (Chisman et al., 1992). Such 
changes also produce considerable stress in workers as jobs are lost and greater 
demands are made on those remaining. Educators need to be sympathetic to 
these stresses and make sure that they are not seen as management agents 
imposing change, but rather as facilitators helping workers come to terms with 
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the new situation and providing constructive assistance in dealing with changed 
work practices. 

In these situations, instruction should focus on skills that will help learners 
cope with their added responsibilities. It should also provide opportunities to 
practice the new skills so that productivity will be maintained. 

EXAMPLE: A credit card company reorganized its staff of telephone 
operators whose job it had been to answer calls from cardholders 
and route them to the appropriate department so that their questions 
could be answered. But due to reductions in the size of each 
department, operators were asked to answer some questions from 
customers themselves. The operators required instruction in 

• skim reading, in order to read a job aid that helps them decide how to 
handle calls; 

• decision making, in order to help them decide when to answer a question 
or forward the call to someone else; 

• communication, in order to understand the customers’ problems and 
resolve them; and 

• writing, in orde’" to allow them to record accurately how calls are handled. 
These were new skills, best learned and practiced in a job-related context. 

INCREASED USE OF LITERACY IN THE WORKPLACE 



• Workplaces use a combination of prose, document, and quantitative 
literacy. 

• Different skills are often used together. 

• Instmction should use realistic simulations to provide a link to the 
way skills are used on the job 



The demands made on the skills of workers are increasing all the time. As 
part of teamwork, planning, and quality control, workers need to be able to 
solve problems that often involve the application of several skills. Literacy in 
the workplace requires a combination of 

• prose literacy (e.g., manuals, newsletters, memos), 

• document literacy (e.g., tables, forms, charts), and 

• quantitative literacy (e.g., calculating, solving problems with numbers). 

Also the different types of skills required — calculation, reading charts or 
other technical material, handling data, problem solving — are often used 
together to perform a workplace task. 

EXAMPLE: In high-performance workplaces, teams of workers 
representing activities at various stages of production often meet in 
what are called “quality assurance teams.” Teams of 4 to 6 workers 
are assigned the task of solving a production problem such as excess 
inventory. Allowing excess inventory to build up in one production 
area may lead to spoilage, breakage, unnecessarily high storage 
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costs, and other problems. To solve problems like this, they will 
have to: 

• read inventory graphs, charts, records, or reports; 

• synthesize all this information; 

• brainstorm for possible solutions; 

• discuss their ideas; and 

• write a brief proposal to management explaining how to solve the 
problem. 

In such a case, classroom job simulation exercises that use the context of a 
realistic production problem can provide the link between the various literacy 
skills and employees’ jobs. 

INCREASE IN TEMPORARY AND PART-TIME WORK 

• Due to the ever-increasing costs of long-term employment commitments 
(such as benefits), employers are less likely to invest in permanent, full- 
time employees. 

• Individual workers will therefore have more major job changes over their 
lifetimes. 

• Temporary workers face higher and more varied literacy requirements, as 
they move from job to job. 

• They will require education in flexible skills such as using resources to 
solve problems and answer questions on the job. 

Another phenomenon with fundamental training implications is the increase 
in temporary, contingent, and part-time work (Bureau of Labor Statistics, 1992; 
Gordon, 1993; Judis, 1994). Employers in the United States are using these 
workers increasirgly as a means of reducing fixed labor costs such as benefits 
and long-term employment commitments. As long-term commitments by 
employers decrease, more workers will have more major job changes 
throughout their careers. Temporary and contingent workers also face higher 
literacy requirements. They must be facile learners who can constantly adjust to 
new sMll demands as their work environment changes. 

Recognizing that temporary and part-time workers may frequently change 
job responsibilities, it might not always be practical to train them in the specifics 
of a particular job task and its corresponding literacy skills. Instead, you should 
help them-develop skills that can have some transfer to other tasks. 

EXAMPLE: One large company that contracted temporary employees 
changed its training program from one that taught employees 
specific computer applications to one that taught them how to locate 
information and solve problems. Rather than just being trained to 
use specific software packages, they were also trained to use 
manuals to find information and apply that information to answer a 
question or solve a problem. 

Having learned to use resources to solve a problem, employees can transfer 
these skills and become more independent as their responsibilities change. 
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WORKER Populations 

LOW LEVELS OF WORKER SKILLS 



• The National Adult Literacy Survey (NALS) shows that one half to 
three fourths of workers in certain job categories perform at the lower 
two levels of the survey. 

• Job literacy requirements are increasingly at Level 3 or above. 

• Workplace instmction should focus on these more demanding job 
tasks, and build in practice of the necessary skills. 

• Opportunities should be provided for long-term education for learners 
at the lowest levels if improvement is expected. 



To give some idea of the actual skills of workers, consider some of the 
results of the recent National Adult Literacy Survey (NALS; National Center for 
Educational Statistics, 1993). The survey uses various realistic tasks in the three 
skill areas of prose literacy (e.g., identifying facts in an article or report), 
document literacy (e.g., finding information in a bar graph, locating points on a 
map), and quantitative literacy (e.g., calculating toted costs of a purchase, 
figuring sales tax). The percentage of adults correctly completing the tasks was ^ 

used to establish task difficulty ratings along a 0-500 scde. Then, each adult ij 

surveyed was given a proficiency score on the same scale according to the tasks ./ 

successfully completed, For example, the task ‘Total a simple bank deposit 
entry” was rated at 191 on the quantitative scale, and adults with a proficiency 
score of 191 would be given that score because they had a high probability 
(80%) of completing that item. These people would then have a steadily 
decreasing probability of success with more difficult tasks that were given 
higher ratings of 250, 300, 350, and so forth. 

In addition to the scale scores, the tasks are further divided into five levels 
of difficulty, ranging from the simple (i.e., tasks with a scale score of 150 are 
assigned to Level 1), to the complex (i.e., tasks at 500 are assigned to Level 5). 

Learners’ proficiency scores are then used to group them into the five levels. 

For example, a learner who achieves a scale score of 250 on the document 
literacy portion of the survey is said to be at Level 2. Thus, this individual has 
an 80% chance of successfully completing Level 2 tasks (e.g., locating an 
intersection on a street map), but somewhat less chance of successfully 
completing higher level tasks (e.g., using a table to determine the pattern in oil 
exports across years. Level 4). Therefore, it is important to remember that when 
a group is described as being at Level 2 on the NALS, it does not mean that 
they are totally incapable of completing higher level tasks, only that the chance 
of successfully doing so is somewhat decreased. 

Data from the NALS show that from one half to three fourths of workers in 
certain job categories (e.g., transportation, service, farming, laborer) perform at 
the lower Levels 1 and 2 on the prose, document, and quantitative scales. In 
addition, surprising numbers of high-level job holders (e.g., managers, 
adininistrators, professionals) also scored at the lower two levels of the survey 
(National Center for Educational Statistics, 1993). Today’s high performance 
workplaces are increasingly requiring tasks at Levels 3, 4, and 5, which call for 
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solving problems, calculating, synthesizing information from several sources, 
and using such a synthesis to m^e decisions. Individuals at Levels 1 and 2 on 
the NALS can expect to face increasing difficulties. Although results from the 
NALS may appear to indicate a serious problem in the worlrfbrce, these results 
do not imply that workers are completely unable to perform some parts of then 
jobs, only that they may not be able to solve more difficult problems accurately 
at least 80% of the time. Also, familiarity with a situation will usually allow a 
worker to perform at a higher level than with an unfamiliar problem. But, in this 
time of changing workplaces, many workers are finding themselves dealing 
with new situations on a daily basis. 

EXAMPLE: In. a food manufacturing plant, management began to 
place an increased emphasis on quality control as a means to 
improve the plant’s efficiency, but due to decreased sales, cutbacks 
were made in the number of supervisors. Workers were asked to 
take over the task of calculating the efficiency of the production line 
for their shift, a task formerly performed by supervisors. This 
involved tracking the number of hours the line actually ran, 
calculating with these numbers, converting the answer to a 
percentage, and recording it on a chart. To help workers take over 
this responsibility, a program was developed that used realistic 
scenarios and allowed for practice time. In addition, most learners 
used published math workbooks to brush up on their skills with 
percentages. 

In high performance workplaces, workers are expected to perform more 
demanding job tasks such as the ones described in the example above. To make 
the most effective use of resources, literacy education should focus on the more 
demanding job tasks, with opportunities to practice the required skills and 
improve workers’ chances of solving challenging problems successfully. 

A test such as the Test of Applied Literacy Skills (TALS, which is derived 
from the NALS) or the CASAS Life Skills Assessment, can be used as a 
diagnostic tool to determine the actual skill levels of learners. Using one of 
these tests as opposed to a more general standardized test (e.g., the Test of 
Adult Basic Education, TABE) provides three important advantages. First, such 
tests use more realistic tasks that are similar to ones that workers may encounter 
in their jobs. In addition, the results will point out skill areas that may require 
extra practice. Finally, the results will help to identify learners who have less 
skill and may need more basic skills education before they are ready to enter 
job-related skills training. In this manner, low-level learners can receive long- 
term instruction that will enhance their skills more effectively than would a 
shorter, technical, or basic skills class. 
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WORKERS WITH LITTLE ENGLISH 



• Workers with little English are over-represented in lower-skilled jobs, 
less likely to be provided with education from employers, and 
unemployed more often and have greater needs for education due to 
turnover. 

• Such workers do not all have the same need for instruction in reading 
and writing, listening and speaking. 

• Workers from other cultures may need help in making the shift to 
American workplace behaviors, attitudes, and expectations. 

• Workers with little English are likely to learn more quickly if they are 
taught in the context of the job and outside interests. 



Immigrants and their children, especially Hispanics and Asians, are 
projected to comprise an ever-increasing proportion of our ’ 'orkforce as they 
will our general population (Bureau of Labor Statistics, 1992). Currently, the 
Hispanic population is over-represented in lower skilled jobs that often do not 
require a high school diploma (Boisjoly & Duncan, 1994). In addition, 
Hispanics tend not to receive workplace education, and they tend to have less 
access to jobs that require some form of training to obtain (Amirault, 1992). 
Lack of literacy and communication skills causes Hispanics to suffer more 
unemployment in recessions. This, too, may suggest more need for retraining 
due to greater turnover (Boisjoly & Duncan, 1994). 

Workers whose first language is not English are, of course, not a uniform 
group. Some have very httle formal education in their first language, others 
have college degrees. Some may speak English moderately well, but have 
difficulty reading it, while others know very little English at all. Factors such as 
these will produce very different rates of English literacy acquisition and 
therefore vastly different requirements for instruction. For example, one ESL 
class may concentrate on the skills of listening and speaking to establish basic 
copnmunication in the workplace, while another class teaches workers how to 
read safety rules and write brief memos. 

For workers from other cultures, an additional factor needs to be addressed. 
They may briiig attitudes and behaviors about work, bosses, teams, time- 
keeping, and other issues different from those prevailing in an American 
workplace. They may, for example, be accustomed to taking orders without 
question or come from a culture in which public criticism involves serious loss 
of face, and so have difficulty adjusting to a teamwork approach, which 
involves volunteering ideas and discussing problems and their solutions. Body 
language is an important component in personal interactions, and it too differs 
from country to country. Such cultural differences need to be brought out in the 
open during class, so that workers can be encouraged and assisted in making 
the necessary cultural shift to accommodate to American workplace behaviors, 
attitudes, and expectations. 

To facilitate workers’ abihty to learn English and to do their jobs, language 
should be taught within the context of the job and outside interests. In addition 
to using published workbooks, relevant vocabulary and grammatical 
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constructions should be defined and discussed with the learners using realistic 
scenarios accompanied, whenever possible, by visual aids such as the 
equipment or print materials used on the job. 

EXAMPLE: In a hospital, environmental services employees with little 
English were taught the names of safety clothing (e.g., mask, 
gloves, safety glasses) and when to use them. Actual equipment was 
brought into the classroom to use in role-play situations similar to 
those met on the job, so that the objects, their names, and their uses 
could be clearly associated. The class then discussed tlie importance 
of wearing safety clothing, to reinforce the role-play scenarios and 
provide fiuther practice wiA the new vocabulary. 

Similar techniques could be used with objects or printed materials that relate 
to outside interests such as hobbies, or to life skills such as paying bills and 
performing routine errands. It is worth noting that, for many ESL learners, 
workplace literacy classes may be their first exposure to written English in a 
learning environment. 



Characteristics of Effective programs 

EFFECTIVE WORKPLACE LITERACY PROGRAMS ARE CHARACTERIZED BY 

• multiple strands of instruction that address learner needs and allow a 
succession of learning experiences for long-term education; 

• the involvement of all major stakeholders, including workers, 
supervisors, managers, and educators; and 

• custom-designed curriculum that integrates basic skills instruction with 
workplace applications and materials. 

MULThSTRAND PROGRAMS 



• Multi-Strand programs should address the needs of diverse 
populations on a long-term basis. 

• They should also address the needs of low-level learners by providing 
a sequence of learning experiences from basic to mote advanced 
instruction. 

• A multi-strand program can expand from small beginnings to meet the 
needs of all workers. 



Multi-strand programs have two major advantages over general education 
programs or short-term, “quick-fix” training. First, programs that have multiple 
strands can address the concerns of various populations such as learners whose 
skill levels and job requirements differ. For example, one strand may teach 
quality control workers to use averages as part of statistical process control, 
while another strand may assist workers wiA obtaining their GED certificate. 
Multi-strand programs can also be implemented gradually, allowing workers to 
improve their basic skills, move into higher level technical training, and 
eventually complete technical education courses at a community college. 
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A second major advantage is the assistance that multi-strand programs can 
provide for low-level learners. Through testing, low-level learners can be 
identified and encouraged to participate in general education to prepare them for 
major job changes or more advanced literacy classes. Thus, multiple strands can 
allow a program to be implemented gradudly, focusing on critical issues first 
and thei\ expanding to meet the needs of all workers. 

■'V., 

EXAMPLE: A manufacturing plant’s literacy program began with 
general education for two reasons. First, offering general education 
was less demanding for the instructors, since it relies primarily on 
published materials. More important though, this strand satisfied the 
workers’ concern that it would provide personal benefits such as 
obtaining a GED certificate. Later, more strands were added that 
addressed both management’s and workers’ concerns, such as 
completing time cards and insurance claim forms correctly. 

INVOLVEMENT OF MAJOR STAKEHOLDERS 



• The support of management, workers, and union members is 
important to the success of any workplace program. 

• A good way to achieve this is to form an advisory committee made up 
of representatives from the key groups in the workplace. 

• Involvement in the planning stages increases the likelihood that 
stakeholders will buy in to the program. 



The support of management, workers, and union members is important to 
the success of your program. Creating an advisory committee made up of 
representatives from the key groups in the workplace is a good way to decide 
how a program will proceed. You will hear the views of the other partners, and 
the members of the committee will also hear from each other. In this situation, 
they are more likely to realize the need for consensus than if each group meets 
separately with the program planners,-or worse, is left out of the process 
altogether. 

As an educator, you may talk first to management because of a perceived 
problem that is costing money. For example, the organization could be 
undergoing a major change such as the introduction of new technology, and 
employees may be having difficulty adjusting to the new skill requirements. 
You will need to identify specific problems and obtain information on their cost 
to the organization, so that instruction can be custom designed to address these 
concerns. 

It is also important to discover workers’ needs and concerns, which may be 
different from management’s, and encourage them to buy into the program. In a 
union setting, find out about union goals by talking with representatives, and 
then involve them in program planning at the outset. If the union has 
educational goals involving general education and personal development (e.g., a 
need for GED instruction or a course on managing personal finances), strands 
can be implemented to meet these goals. 
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In a non-union setting, ask if there are appointed representatives who can 
discuss workers’ needs. If there are no such representatives, try to meet with 
workers and get to know them informally, perhaps by talking to people in the 
employee cafeteria during break times. Then approach sever^ people who are 
willing to discuss workers’ needs in greater detail. You migk also ask 
supervisors whom they would recommend that you approach. In either non- 
union or union workplaces, it is also possible to survey all employees about 
their interests, and develop one or more strands to address these. 

EXAMPLE: In the manufacturing plant mentioned earlier, the program 
began with a general education strand that included GED 
preparation. This program allowed workers’ individual needs to be 
addressed. Since participation was voluntary, starting with this 
strand established a reputation for being open and responsive to 
workers, an effective tool for recruiting additional learners. Within a 
short time, a second strand was added that provided instruction in 
job-related skills for one of the most common production jobs in the 
plant. Later, after these two strands were established, workers were 
surveyed concerning their general interests. In this survey, 
employees expressed an interest in learning more about their health 
insurance benefits, especially how to complete claim forms. A brief 
course on this topic was developed and several sessions were 
offered to accommodate different schedules. Thus, concerns of both 
workers and management were met in a timely manner. 

Custom-designed curriculum 



• takes learner interests into account, 

• is based on an analysis of job tasks and skills, and 

• uses materials from workers’ jobs. 



The education program that you develop will use some off-the-shelf 
materials for strands such as genertd education for low-level learners and GED 
preparation, but it will also incorporate custom-designed instruction. 
Customizing the curriculum will allow you to meet the needs of both 
management and workers. 

Using program resources to meet the workers’ interests allows them to take 
an early and active part in planning and developing the program. This will 
eliminate their concern that management is trying to “fix” a problem workforce, 
and instead communicate a desire to invest in people. Learners will buy into the 
program, increasing enrollment and retention. 

Some of the custom-designed strands of the program will be based on task 
analysis. This involves interviewing and observing employees who are expert at 
their jobs to document job tasks and supporting skills. While more time- 
consuming than using pre-packaged educational materials, instruction that uses 
material from workers’ jobs will provide i diiect link between the skill and the 
job, with immediate reinforcement of skills learned, thus increasing practice 
time and speeding up the learning process. 



NATIONAL CENTER ON ADULT LITERACY 



o 

ERIC 



28 



15 





EXAMPLE: In a food manufacturing plant, task analysis was used to 
discover the skills necessary to perform each job. Print materials 
were also collected and later used in class as the basis for custom- 
desired class activities. In the plant, task analysis of the production 
line jobs revealed a problem with reading manuals that contained 
piping diagrams. Instruction was developed that taught workers 
how to read and interpret these diagrams. This enabled workers to 
use the production manuals to solve problems rather than wasting 
time waiting to ask supervisors. As a result, down time was 
decreased and the overall efficiency of the plant increased. 

Chapter Summary 

Over the past two decades, research has revealed important information that 
characterizes literacy education in the workplace. Organizations face multiple 
challenges with regard to workers’ skills and the ever-increasing demands of 
jobs. Programs that have multiple strands (e.g., general education for low-level 
learners, GED preparation, job-related skills classes) are able to address these 
issues most effectively. Improvement requires significant practice time, often 
more than is available in classes. Programs have compensated for this lack of 
time by providing additional practice time at home and by using materials that 
are job-related to gain practice as learners perform their jobs. To ensure that 
skills will have an impact on job performance, instruction should be targeted to 
specific job tasks. Besides increasing practice time, using job-related materials 
increases learners’ ability to transfer slalls directly to the job, thereby increasing 
productivity. Since new skills are lost without regular practice, instruction 
should also be timed to correspond closely to when the new skills will be used 
on the job, or else learning may be lost. 

The changing labor market also plays a significant role in the nature of 
workplace literacy programs. Labor market trends such as the shift to more 
service sector jobs, high performance workplaces, increased use of literacy, and 
increasing temporary and part-time work ^1 influence the need for workplace 
education. Instructors and program developers should strive to make programs 
relevant to workers’ needs such as the need for flexible skills that will apply to a 
variety of jobs, and the need for increased skill in high-performance workplaces 
that use techniques such as increased quality control. Programs should also 
address such concerns as low levels of worker skills and immigrants with little 
English. Long-term programs that begin with basic reading, document, and 
quantitative skills, and advance to job-related skills education and beyond will 
help learners improve productivity and address personal needs such as 
improving chances for advancement. 

As programs are planned and implemented, educators should concern 
themselves with three important workplace education needs. These are 
involving stakeholders in the early stages of planning, developing programs 
with multiple strands to address multiple needs, and custom-designing the 
curriculum to suit particular jobs and skill areas. Details about planning and 
analyzing the needs of the workplace will be discussed in Chapter Two. 
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Chapter Two 



PLANNING A WORKPLACE LITERACY PROGRAM 

This chapter provides guidance on the following topics: 

• analyzing the need for literacy education, 

• addressing the needs of various groups within a workplace, 

• using non-training solutions to increase instructional effectiveness, 

• locating sources of funding and resources, 

• planning the logistics of a program, and 

• developing a shared vision. 

Effective Workplace Literacy Programs 

A growing body of research has identified certain characteristics associated 
with effective worl^lace literacy programs. Relatively few workplace programs 
meet all these criteria, but the degree to which they can be met appears directly 
related to their success: 



• Multi-strand programs are an effective way to meet the demands of the 

diverse populations within a workplace. 

• Custom-designed materials integrate basic skills instruction with 

workplace instruction. 

• Forming an advisory committee involves major stakeholders in the 

planning process. 

• Planning for long-term educational goals allows workers — and the 
program — to grow. 



It is important to realize that we face several literacy issues in the workplace 
and not just one. Therefore, no single class or course can meet the demands of 
the diverse populations within a workplace. By using a multi-strand approach 
with several different types of courses and strings of educational experiences, 
you can meet more needs and build on previous learning experiences. The aim 
of the program, which needs to be agreed on by all stakeholders, should be to 
address long-term educational goals wliile satisfying short-term needs. 

EXAMPLE: At an auto manufacturing plant, short courses on safety 
procedures for new equipment were offered to learners. After 
completing these courses, learners were able to select from 
additional educational experiences such as GED or technical 
courses. 

Another characteristic of effective programs is using custom-designed 
materials that integrate basic skills instruction with workplace applications. 
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When a person cannot transfer learning to real-world situations, he or she 
cannot continue to practice what has been learned. Custom-designed materials 
link instruction directly to workplace demands and allow the learner to build on 
previous knowledge. 

Having an advisory committee play an active and consistent role in 
developing and monitoring the program is important to its long-term 
effectiveness. The committee can introduce you to the workforce, provide 
valuable information and insight about the worl^lace, determine what resources 
are available, assist you in deciding which groups of workers to target first, and 
help gather information and recruit students. Depending on the organization, 
your advisory committee might include employees, administrators, 
representatives from unions or other partner organizations, departmental 
supervisors, and members of the company training department. 

Since few programs can initially meet all these criteria, try to emphasize first 
what you can accommodate in your own program. You might have to start 
small, but it is important to build a vision for your program’s future. Ask 
employees to suggest future courses that they would be interesting in taking. 
Also, be aware of cunent and future educational needs that are not being 
addressed. By building on employees’ interests and the future needs of the 
company, you v/*ll develop a stronger program. 

EXAMPLE; At the auto manufacturing plant mentioned earlier, the 
literacy program began with short courses on safety procedures. 
Instmctors learned from the workers attending these classes about 
other needs and interests — such as a GED class, which was started 
soon after. The plant was also setting up work teams, requiring 
better communication skills, and introducing statistical process 
control. To meet this new need, both management and union wanted 
classes on writing memos and meeting minutes, and on basic math 
leading toward cdculating averages and plotting graphs. Thus, over 
a peri(^ of a few years, the program grew from its initial one course 
to a true multi-strand program, satisfying the needs of that 
workplace. 

PLANNING AHEAD 

In determining the specifics of your program, the following activities will 
help you to plan ahead. You should first an^yze the needs and wants of both 
the company and the employees. Find out where the problems are — are there 
communication problems between management and employees, are there skills 
that workers need to improve? After you have found out where needs are, you 
then determine whether a literacy program will help meet those needs. If such 
education will help, then you have to decide what types of courses will be 
offered and to whom they will be offered. Also analyze what types of non- 
training solutions will be important to the success of your program. 

Finding money for workplace literacy programs can be a challenge. Even 
when the company or union funds them, it is important to look for other 
sources of money. You can also look for opportunities to share resources with 
other company departments or other organizations. By planning ahead, you can 
build a more effective and successful workplace literacy program. 
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Analyzing the Need for Education 



• “Where is the pain ?’' — ^what problems are causing real concern for the 
employees and company. 

• Gather information through interviews and surveys, and by observing 
daily operations in the workplace. 

•f. 

• Determine which areas of company performance need improvement: 
find out the differences between where the company and the workers 
are and where they want or need to be. 

• Define which problems are most pressing and determine whether they 
can be solved through education. 



Before planning a program, you should analyze the workplace situation in 
general. Find out the differences between where the company and the workers 
are and where they want or need to be. What are the needs and wants of the 
employees, the company, and the union? What do the stakeholders expect your 
program to achieve? For example, some employees may want to raise their 
general educational level in order to obtain a GED certificate. The company may 
want all workers to be able to carry out statistical process control procedures. 
Or a union may want to increase their members’ knowledge about health 
benefits. 

You can find out this information by conducting a needs analysis. A needs 
analysis can take many different forms and can vary in depth and 
comprehensiveness. The type of needs analysis you do is often dictated by the 
amount of time and money you have available. A needs analysis is done by 
arranging focus groups, interviewing people one on one, asking people to 
complete surveys, and observing daily operations in the workplace. 

Suppose the company wants to raise the job skills of the workforce. By 
interviewing a broad range of employees, managers, and supervisors, you can 
find whether performance problems exist and what they are. By asking “Where 
is the pain?” — ^what problems are causing real concern for the employees and 
company — ^you can then assess whether a literacy program will help solve the 
problems. 

These are all things you should consider when doing a needs analysis. 

• Are there problems witli manuals not matching manufacturing 
procedures? 

• Do employees need extra assistance with completing paperwork? 

• Are there accidents caused by the inability to read directions? 

• Is management not communicating effectively with employees? 

• Do employees have appropriate workspaces? 

• Is there a lack of motivation or incentive among employees? 

After determining what the needs are, you must then define v/hich problems 
are most pressing and decide whether they can be solved through education. An 
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advisory committee can help guide this process by helping you become familiar 
with both employees and the company. 

When doing a needs analysis, you should get all stakeholders involved in 
the process from the beginning. They can provide valuable information about 
the workplace and its needs. They can also help with analyzing problems and 
recommending solutions for them. It is important to analyze the job tasks in the 
workplace. During interviews and observations, you document the tasks 
involved in doing a particular job. At the same time, you collect samples of 
printed materials and discuss them with employees and supervisors. These 
materials will later form the basis of yoin custom-designed curriculum. (See 
Chapter 3 for more information on this topic.) 

Conducting a needs analysis is an important part of planning a program. As 
a program planner, you should make smre that you are providing a service that 
is needed. Sometimes there are other problems such as conflicts between 
management and workers, an error in the manufacturing process, or an 
ineffective organizational structure. A literacy program will not help solve these 
problems. So you should not assume that low literacy levels are causing 
problems, even when management tells you so. You need to investigate the 
situation thoroughly before planning any program or course. 

EXAMPLE: At one small manufacturing company, a literacy instmctor 
was hired to work with the maintenance staff. The company’s 
management thought the employees were having literacy-related 
problems since they were not completing a certain form properly. 

The company had recently changed its policy, and employees now 
had to complete a form to request certain machine parts. But after 
testing these employees, the instmctor found out that the problems 
were not literacy related. The employees were in fact unhappy with 
the management’s lack of communication about a new policy and 
they were deliberately not completing the forms correctly. However, 
those workers did later report other sorts of literacy areas in which 
they did want help. 

When making reconunendations for changes, be aware that you may have to 
deal with corporate politics. Companies are not often responsive to outsiders 
coming in and making suggestions for improvement. From the beginning, you 
need to make it clear that you are doing this to determine their literacy needs. 
You may also be helping the company by preventing unnecessary training 
programs. You should periodically re-evaluate the need for future educational 
programs. 
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Designing the Program 



• Address the needs of both the employees and the company. 

• Create a balance of workplace demands and learner interests. 

• Offer multi-strand programs to meet the needs of various workplace 
groups. 

• Involve workers in developing and recruiting for your program. 



Designing a customized program that integrates workplace demands with 
other types of educational skills will help better meet the needs of both the 
employees and the employer. Custom-designed programs can help cut hidden 
costs associated with low-level literacy within a company. Reworking, 
retraining, and accidents can be very costly. The money saved can often more 
than pay for a literacy program. Also, if you present the specific needs and how 
they will be solved, more people in the company will be able to see the 
relevance of instruction that is diluted at real problems. 

It is important to balance the program by addressing the employee’s outside 
needs and interests as well as workplace needs. Combining job skills 
instruction with life skill materials such as medical information or appliance 
repair makes the instruction more relevant and more effective. For example, if 
your program has a course designed to teach employees to use workplace charts 
and graphs, you could also provide instruction in using charts and graphs for 
items used at home such as nutritional information charts. This will maximize 
the effectiveness of your program by allowing learners to extend and practice 
their skills. It is important to find out what specific types of education 
employees are interested in and integrate that education into your program. By 
offering classes that address employees’ goals and interests, you will directly 
address their personal needs and encourage them to apply their new skills in 
other areas. Gaining employees’ buy-in for your program will be important to 
its future success. They will help you recruit other students and make it a more 
effective program. 

Increasingly, programs in business and industry are becoming multi-strand 
programs. In such programs, one instructional strand might address the needs 
of English as Second Language learners, another strand might help learners 
prepare for GED certificates, and additional strands might help high school 
graduates brush up on skills before preparing for technical training. The 
instructional format may vary from structured classes to small group 
instruction, to computer-guided instruction, to individual tutoring. Some 
learners will, over several years, move from one strand to others. 

EXAMPLE: In a multi-strand program for correctional officers at a 
women’s prison facility, instruction was delivered in three strands. 

The first strand was designed to help officers improve their report 
writing skills, so that reports on offenders could become more 



34 



NATIONALCENTER ON ADULT LITERACY 21 



BEST COPY AVAILABLE 




effective at disciplinary hearings. The second strand focused on 
promotions support and addressed the employee population 
interested in preparing for job advancement. This course helped 
learners prepare for advancement as well as upgrade their skills for 
other post-secondary instruction. A third strand addressed family 
literacy skills and taught parents how to read and share books with 
their children. This encouraged learners to maintain and extend their 
own literacy gains while helping their children develop skills at 
home. 

Employees are not passive receivers of instraction — they are partners. They 
can become a valuable resource for helping you plan, develop, and evaluate a 
program. They can tell you how things are really done at the company, where 
the problems are occurring, what types of education are needed, and what 
courses would be valuable to them as individuals. They are also a valuable 
resource for publicizing and recruiting, as well as for evaluating a course or 
program. 

EXAMPLE: In a literacy program at a manufacturing company, 
employees helped gather data to plan courses, helped develop the 
curriculum, helped delivered the instruction, and evaluated the 
program. Employees contributed kiiowledge based on their job 
responsibilities and expertise. Course participants were especidly 
helpful in specifying which areas of instraction were not related to 
the actual jobs or which job practices did not follow company 
instructions. They also suggested course revisions through 
interviews and surveys. Their assistance not only helped improve 
their own instraction, it also helped improve the design of future 
courses. Since they were involved in course development, they 
bought into the program and helped recruit other students. 

Teachers and educational cocrdinators developing workplace literacy 
programs must confront many issues. However, if you listen carefully, you 
will hear both employees and management saying what problems and desires 
are most pressing or persistent. This will help you determine what types of 
classes to offer. Your goal is to create courses that address the actual needs and 
goals of the employee and employer. 



Addressing the needs and Goals of various Groups 



• Multiple program strands are needed to address multiple problems. 

• Improvement takes significant learner practice time, and varies greatly 
depending on the learners and the material. 



When designing a program, it is important to realize that we face several 
literacy problems in the workplace, not just one. The person who cannot read at 
all requires a different type of support from than the high school graduate who 
is unable to meet the new reading demands of his job. People educated in a 
foreign language and not speaking much English require another type of 
support. Providing the same services and programs to such different clients 
m^es no sense, and yet it sometimes occurs. (A program just starting out. 
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however, may have limited resources and a low enrollment, and therefore be 
obliged to teach mixed groups. In such classes, the teacher must find ways of 
catering to all needs, and could include individual work and large and small 
group activities as appropriate.) 

You will find that there are no magic pills in terms of improving literacy 
skills, and the time needed to make considerable improvement varies greatly. 
Factors like the literacy levels of students entering a program, the complexity of 
material being taught, and the speed at which students are able to leam all affect 
the amount of time needed for various groups. We will now consider several 
groups in the workforce and their possible goals and needs. 

HIGH SCHOOL GRADUATES 

Often high school graduates who have been out of school for several years 
may need a refresher before receiving further training. Some may need to brush 
up on their reading or math skills due to new demands on the job. These needs 
should be taken into consideration when designing a program. Short, targeted 
courses may be the best answer. 

HIGH SCHOOL NON GRADUATES 

People who did not graduate from high school may be interested in 
obtaining their General Educational Development (GED) certificate. These 
people often need to bmsh up on basic education skills before doing this. Some 
may also be interested in furthering their education beyond high school. 
Providing information about two-year degrees and other educational 
opportunities should be part of instruction. 

PART-TIME EMPLOYEES 

Part-time employees often have specific educational needs but may not be 
included in company-sponsored training. Many part-time employees are former 
full-time employees who have been affected by downsizing or a shrinking job 
market. So it is important to address the specific job-related needs of part-time 
employees, and if relevant, provide training in job search strategies. Linkage to 
community-supported programs can also be included. 

VERY LOW LEVEL LITERATES 

One of the conunon myths about workplace literacy programs is that all the 
students are functioning at very low levels. This is not the case — less than 3% 
of adults read at or below a second-grade level. However, if testing reveals a 
significant number of adult very low-level literates in your workplace, you will 
need to decide if you can help them or need to refer them to other programs. For 
this group to improve to the point of being able to read a manual or a 
newspaper, it takes several hundred hours of practice and often one-on-one 
tutoring. For some severely learning disabled adults, comprehending high 
school level training materia may simply be out of reach. Working with local 
literacy providers or setting up a peer tutoring service in the workplace can 
provide an opportunity for you to find out who in this group can be helped by 
you. If you are considering a peer tutoring system, you need to check whether 
there will be any problem with learner confidentiality or possible social stigma 
for those in need of help. 
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Others in the workplace may be middle-level literates who have only 
recently encountered demands beyond their abilities. Some adults can work 
their way through simple sentences but are not prepared to handle extended 
written instruction, manuals, or new training material. This group can make 
fairly rapid gains in a hundred hours or so if instruction is focused and 
effective. It is important to use job materials so learners can practice their skills 
and knowledge daily at work. Linking to home interests can further extend 
practice. 

To help these people, look at what the students really “need to know” for 
the job and “want to know” for home use. Teaching material that is “nice to 
know” adds class time and frustration. Be flexible and creative in the range of 
methods that your teachers use. Provide these students with materials that they 
can keep so that they can refer back to instruction and notes. It is also very 
helpful to plan a definite scope and sequence to all instmction, and create many 
short-term visible goals. 

t 

ENGLISH AS A SECOND LANGUAGE 

Increasingly, many of the people entering the workplace are not native 
speakers of English. They can often make rapid gains if their educational needs 
are met directly. Most of the suggestions made for low-level literates are 
applicable to ESL learners. In addition, you should determine specifically what 
the actual language needs are. Find out the areas of oral or print communication 
that call for the most immediate help. You can design many customized 
materials for ESL learners, such as how to complete a job application, how to 
read safety instructions, and how to use the telephone. Using role playing for 
specific situations such as how to start a conversation in English is also helpful. 
Role play and discussion of cultural differences between the members of the 
class and American society can help ease tensions by bringing such issues out 
in the ojwn. Group activities calling for simple oral language use pay dividends 
in later literacy development. 

In addition, you can provide these students with simple texts and 
workbooks, so they have something to take home for further practice. You 
should encourage them to share their knowledge with their families. When 
working with ESL learners, be aware that some of the students may not have 
had a great deal of education in their first language. This often makes it more 
difficult to learn a second language. Others, however, may have had a high 
level of education in their first language. Find out all you can and try not to 
make assumptions based on English mastery alone. 

OTHER LEARNER NEEDS 

Sometimes your program does not currently have the resources to address 
all the students’ needs. Learners might need something other than what your 
program offers. When working with various groups of students, be aware of 
other resources in your area. Have information on local colleges, technical 
schools, and GED programs. It also helps to have knowledge of resources such 
as social services, child care, care for aged, and other programs that might be 
needed to ftee employees to attend class. 



37 



o 

ERIC 



?4 



PRACTICE GUIDE PG9G-01 




Designing Non-Training Solutions 



• Non-training solutions include providing job aids, redesigning forms, 
and rewriting instructions. 

• But be careful — employers will not always be open to these types of 
changes. 



While analyzing the needs of a worlqplace, you should determine if there are 
non-training solutions that would help increase the effectiveness of the 
program. Some non-training solutions include providing job aids for employees 
to help them remember tasks, redesigning forms to maJce them easier to use, 
rewriting instructions to make them clearer, and performing detailed task 
analyses to see if job design might be the problem. 

EXAMPLE: Literacy instructors at a women’s prison facility designed 
several job aids for correctional officers and counselors. One job 
aid, a 3x5 note card, was developed for taking notes to write 
conduct reports. It was designed to organize details quickly and 
completely. The job aid included prompts that acted as memory aids 
to the writer. For example, words such as “who, when, wimesses, 
evidence” were listed in a small notebook, so that officers could 
quickly jot down pertinent information. The back of the job aid 
consisted of a mini-glossary of words often found in these types of 
reports. Correctional officers helped develop these glossaries 
themselves based on words that proved to be spelling or usage 
problems. 

In order to identify which literacy and communication skills are causing 
problems, look at the entire situation. Build the relationship necessary to be able 
to make recommendations. Consider what non-training solutions might 
accompany or substitute for instruction and the cost of developing and 
providing these solutions. However, you should approach the issue of non- 
training solutions carefully. Employers will not always be open to these types 
of changes. For example, if a company spends hundreds of thousands of 
dollars on a manual to train employees to use a computer system, they might not 
be willing to have that manual rewritten and reproduced. They might, however, 
agree to developing a job aid to accompany the manual. 
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Locating Resources and Planning Logistics 



• Funding sources include state and federal grants, partnerships with 
libraries and colleges, workplace unions, and management. 

• Program teachers are a cmcial element and may need help with using 
job-related curriculum and becoming sensitive to the workplace 
culture. Above all, they must be flexible. 

• Look for ways to share resources with unions, companies, and 
training departments. 

• Discuss the scheduling of classes around the company’s activities to 
achieve maximum cooperation. 



LOCATING FUNDING SOURCES 

There are several options for funding programs. Some external sources are 
state level grants, federal literacy grants, and partnerships with local libraries, 
colleges, and volunteer organizations. To get more information on federal and 
state grants, contact the U.S. Department of Education or your state department 
of adult or vocational education. 

Many unions also offer viable funding sources because they recognize that 
technology is driving jobs and promotions, and that seniority no longer offers 
sufficient job protection. Larger unions are working to be model educational 
providers to their membership. Smaller unions and locals often work with 
management to develop local educational and support progr ams 

EXAMPLE: A literacy program at a textile manufacturer is funded 
from various sources. The manufacturer provides office and 
classroom space and supplies. The program also shares resources 
from other departments such as computer-based instruction 
programs and workplace manuals. The textile union funds the 
instmctor’s salary and the cost of materials such as books and an 
independent graiit funds a portion of the family literacy strand 
offered at the company. 

Use your advisory team to explore various funding possibilities. Consider 
ways to get multiple sources of funding. When considering funding sources, 
keep in mind that outside funds may need 100% employer match; federal 
funding is usually seed money to start programs; and there may be some delay 
in obtaining funds. Whatever funding path the company takes, make sure that 
you work with the person in direct budgetary control from start to finish. 

Depending on your specific situation, you may have to write a proposal to 
fund your workplace literacy program. The proposal will define the program in 
terms of time, resources, support, and finances. It is important that your 
proposal be as clear and detail^ as possible. 

The proposal should include objectives and rationales for the program, a 
step-by-step outline of the project, an evaluation plan, a management plan 
detailing logistics and a timeline, a budget, and a listing of personnel involved. 
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Funding agencies usually have specific guidelines for how proposals should be 
written, but clear organization, concise writing, and a minimum of jargon form 
the foundation of a good proposal. 

STAFF DEVELOPMENT 

Program teachers are a crucial element of any workplace literacy program. 
Once instruction has started, they are the ones in day-to-day contact with the 
workers, and a program can succeed or fail depending on how well the teachers 
can interact with the learners in their classes. 

Because workplace literacy is a growing area, many teachers in workplace 
programs are recruited from adult basic education or school settings, and have 
no experience of how to teach job-related curriculum. They may be accustomed 
to being the subject matter expert in an academic classroom, and so have some 
difficulty coming to terms with the shared responsibility of many workplace 
classes, where the learners are content experts on their jobs and teachers need to 
learn as well as teach. Another aspect of teaching in a -orkplace is sensitivity to 
the workplace climate and culture, and respect for the individuals as adults. 
Here again, the teacher can learn much from the workers, by listening carefully 
and not making assumptions about them or their jobs. In this way, a 
relationship of confidence can be established, which will facilitate teaching and 
learning. 

Much of what a workplace instructor needs to know and do can be summed 
up in one word: flexibility. Teachers should always be prepared to learn from 
the workers in the class and adapt instruction to ±eir immediate needs. More 
generally, a teacher’s view of the whole workplace program should be similar 
to that of a program developer. This handbook can be a useful resource for 
training new workplace educators. The Appendix contains a series of 
sununaries suitable for use on an overhead projector, to facilitate the task of a 
trainer. 

However well a program has assessed the needs of a workplace, and 
however good a curriculum has been developed to meet those needs, it is 
ultimately the instructor in the classroom who determines the success of the 
program. It is therefore vital that a program selects good teachers and prepares 
them well for their role in the workplace. 

ALLOCATING RESOURCES 

Sufficient resomces and their sensible use are very important in establishing 
a smoothly running program. Resources include money and personnel, but also 
relationships with key officials, facilities, supplies, storage space, and ways of 
advertising your services to employees. 

When planning your program, discuss with both company and employee 
representatives the possibility of using shared resources. Here are some 
suggestions: 

• Maintain relationships with the various departments of companies and call 
on them for ideas for solving problems. 

• Negotiate with unions to draw on their expertise and to use their facilities. 
Some unions have classrooms that may be available to your program. 
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• Investigate using loaned resources. For example, having the company’s 
computer programming department design your computer-based 
instruction may save money. 

• Discuss in detail the facilities you will have to work in — it is very helpful 
to be able to stay in one area and not have your class locations shifted 
around. 

• Find out what supplies and what trainers you can share with other 
departments, and negotiate with them. 

• Arrange for some form of secure location or storage space so that supplies 
are not appropriated by other departments. 

• Use the company’s advertising department to design advertisements for 
the company newspaper. 

• Have students contribute articles about the program to the company 
newspaper. 

PLANNING PROGRAM SCHEDULES 

Planning the logistics of a program can minimize potential problems with 
scheduling. Consider the company’s production schedule and how you might 
incorporate classes into it. The classes must be as convenient as possible for 
everyone without interfering with business. Options include scheduling classes 
before and after shifts, and developing take-home lessons and computer-guided 
instruction to allow make-up time. 

Other considerations include starting courses when the company has a 
slower production schedule. You could include whole departments while a line 
is temporarily closed as opposed to taking a few members from several different 
departments and allowing work to continue. Discuss the options with 
management, supervisors, teachers, and union officials before developing a 
course schedule. The most important thing is to be flexible. 

BUILDING Effective Relationships 

To build an effective literacy program, you must 

• gain the support of management and employees, and 

• maintain positive relationships with management, employees, and union 
representatives. 

GAINING MANAGEMENT AND EMPLOYEE SUPPORT 



• Balance education and productivity needs. 

• Involve management and employees in program development. 

• Decide together what needs can be addressed first with the time, 
money, and resources available. 

• Develop a shared vision for the future. 
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Educators developing workplace literacy programs must confront several 
initial questions. What are good ways to involve employee leadership in the 
development of literacy programs? How can education and productivity needs 
be balanced? Like management, employees and unions have goals. Some of 
these overlap management goals and some relate to other more general worker 
concerns. Find out about these goals by interviewing employee leaders and 
getting their feedback on program plans. Secure cooperation from top leaders as 
well as plant and department level representatives. And, of course, consult 
contract rules. 

Unions can be a valuable resource. Union members can participate in your 
program by serving on advisory teams, setting up joint program?, ?r providing 
funds. Y ou can also have them recruit students, provide space for classes, and 
suggest classes. Like company management, unions often have specific needs 
and goals. Find out what these are. Make sure you always consult with the 
union about any decisions that are made. When unions are not available, you 
may have a more difficult time identifying employee leaders. But it is important 
to do this since programs without employee support tend to fail. 

It is difficult to design a program to meet everyone’s needs. As a program 
designer, you have the job of determining which educational needs will be met 
initidly, which later, and which not at all. This is a difficult job when you are 
sitting in a room with various representatives who have vested interests in 
different outcomes. For example, human resources may want you to address 
ESL education needs specifically. Union representatives want to maintain 
employee job security and want you to address general education needs. The 
company’s training department may want you to address specific technical 
training for a new production process. Employees may want to learn the general 
use of computers. You have to decide what needs can be addressed first with 
the time, money, and resources you have. This involves negotiation, 
persuasion, and a good deal of explanation about realistic expectations. 

It is important to develop a shared vision among the representatives and 
look for innovative strategies for meeting their needs. For example, you might 
design a multi-strand program that initidly offers computer-based training on 
new production processes. Later strands would include ESL and general basic 
education skills assistance. If time and money limit your start-up options, you 
may have to begin by addressing the most pressing needs first. 

Developing a shared vision for the future is important not only for 
maintaining go<^ relations with various representatives; it also makes a program 
more effective in the long term. If there is no shared vision among the 
representatives, it may be best to agree to provide a more general type of 
program or not have a program at all. 
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MAINTAINING RELATIONS WITH EMPLOYEES, MANAGEMENT, AND 
UNIONS 



• Successful outcomes and satisfied clients are achieved by reaching 
jointly agreed upon and reasonable goals. 

• Constantly review the program’s progress with supervisors, 
managers, and unions. 

• Keep the lines of communication open, if possible through an 
in-house liaison person. 

• Keep accurate records for your program on finance, and learner 
attendance and progress. 



Many workplace classes last only 20 to 30 hours, so it is wise to set up 
well-defined and limited goals from the beginning. In order to capture and retain 
the support of employees, management, and union officials, all partners need to 
see results from your program. Listen to concerns and include them in all the 
logistical considerations of the program. Develop clear program goals and then 
make sure the goals are achieved. You should not promise to deliver more than 
is realistic. Successful outcomes and satisfied clients are achieved by reaching 
jointly agreed upon and reasonable goals. 

Supervisors, managers and unions want to and should feel a part of the 
positive results of a literacy program. It is a good idea to constantly review 
progress of employees in general terms with supervisors without refening to 
individual learners by name. You can also hold recognition ceremonies to 
reward both employees and supervisors. Before doing this, it is important to 
determine whether learner privacy is an issue at your workplace. 

There are advantages to having a dedicated, fully-trained, in-house person 
in charge of the program to assure open lines of communication between 
literacy educators and supervisors. If possible, this person should be in a highly 
visible office, have rapid access to all supervisors, union representatives, the 
advisory team, instructors, and involved management, and have access to 
facilities for electronic mail or other rapid communication. All supervisors and 
managers who are involved in the literacy program should be made aware of the 
classes in progress and those planned for the foture. 

Also, it is important to keep accurate records for your program. You should 
track how much money is spent and what it was used for, how effective the 
instruction was (e.g., pre- and post-test scores), and the dates employees are 
absent and why. You can use these records in later conferences with the 
advisory team or with management. Plan to check regularly with supervisors 
and the joint advisory teams to see if the original needs are being addressed and 
to see if your program is having an impact. 

As a literacy program developer, it is important for you to realize that you 
may be serving in the role of a change advocate or change agent. A change 
advocate supports the need for change; a change agent plans changes and 
attempts to implement them. Change agents try to implement and manage 
change by identifying those that are critical. They analyze each change and 
design a plan to integrate the change successfully. They must also execute and 
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revise that plan as needed (Dormant, 1986, p. 238). Literacy programs are 
often bound up with larger changes going on in a company such as changing 
job descriptions oi downsizing, and, as the program designer, you need to be 
aware of the politics of change. 



Other Suggestions for Designing an Effective Program 

Other elements of an effective literacy program include 

• creating positive relationships with all parties involved, 

• being flexible and creative in solving problems, and 

• making sure your clients and advisory team are satisfied with the 
program. 

Creating a positive, actively involved relationship with employees, 
management, and union officials will help maintain support for the program. By 
meeting regularly with joint advisory teams, you can help make sure your 
clients are satisfied with the directions and outcomes of the instruction. As a 
program designer, you should also understand the need for flexibility. 
Anticipate and plan for problems and find creative solutions. The problems of 
starting a workplace literacy program are often fairly predictable. These 
solutions, if planned ahead of time, can be in place before the need arises. 

Some of the common problems and solutions include the following: 

• Production supervisors have many short-range goals to be met and 
tension arises when smdents need to be in class and a “hot order” has to 
be completed. The solution for this problem revolves around having the 
supervisors “buy into” the long-range goals of the program and 
scheduling classes at times when production is most likely to be slower. 
Instmctor flexibility is also a must. 

• Scheduling classes should include discussions with and decisions from 
your advisory committee as to which classes will be taught (or taught 
first), the number of classes a person can take at a time, and the number 
of people in each class. Waiting lists and open or closed entry classes 
should also be discussed. 

• Confidentiality is a major concern to some smdents taking classes. They 
need to know from the very start that their records will not be shared with 
management, union, or others. However, management can ask you about 
the progress of the class as a whole and is entitled to be given this 
information. The management is also entitled to know if an employee is 
taking or has taken a particular class. 

• Labor/management issues include possible strikes from either unions or 
other employee organizations. You do not want to ask either side to cross 
a picket line, and you should not cross that line yourself. The mined 
relationships that result can kill a program. Some teachers have continued 
classes at neutral sites. 

• Know the policies and relationships between employees and management 
so you will know whether you can go out on the floor for information or 
whether you have to remain in the classroom while others gather 
information for you. 
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• Be very clear from the beginning as to whether the program is an 
employee benefit or a management tool or both. As a benefit, learner 
gods take top priority. If the progr^ is totally funded by management, 
die balance leans much more heavily towards work performance. If 
learners feel deceived about what you are offering, it will seriously injure 
your program. 

• Using the word literacy in the title of your program gives the impression 
of very remedial work. Using a catchy tide or logo avoids the problem. 
(Examples include: “Skills Today for Achievement Tomorrow (ST AT)” 
used by Blue Cross and “The Pepsi Challenge” used by Pepsi.) 

Chapter Summary 

In order to plan a workplace literacy program that is in tune with the needs 
of the workforce, you should organize an advisory committee to a) help you 
learn about the job site, b) interview workers, c) identify basic skills that need 
to be improved, d) determine available resources; and e) help you plan the 
program. After you analyze education needs, you will be able to design a more 
effective program. 

In designing the program, you should consider what types of instruction to 
offer and the needs and goals of various groups. Workforces are most often 
made up of employees with a broad range of skill levels. Non-training solutions 
should also be considered since they will increase the effectiveness of the 
program. 

Locating funding and resource providers is an important part of planning the 
program. Funding tfirough multiple sources often creates a stronger program 
since key groups are directly involved. Getting groups to invest resources is 
another way of getting them involved. Planning the logistics of the program 
early can eliminate potential scheduling problems. Consider the company’s 
production schedule and do your best to avoid inconvenience to company or 
employees. 

Gaining support from management and employee representatives is another 
way to build an effective program. Representatives from both groups should be 
on your advisory committee. They can help you plan and set up the program, 
locate resources and funds, and recruit students. As a program manager, you 
should be aware of issues such as class schedules, confidentiality, 
labor/management issues, workplace policies, and workplace politics. Effective 
management of the program on your part can lead to positive relationships 
among all parties involved. 

As you begin actually developing the pro^am, you will look in more detail 
at analyzing the need for education, determining what types of classes to offer, 
how to design your cuniculum, and how to recruit students. You will learn 
more about these things in Chapter Three. 
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Chapter Three. 



DEVELOPING A WORKPLACE UTERACY PROGRAM 

When developing a successful workplace literacy program, it is important to 
consider what tyres of performance problems exist, what types of curriculum 
you will use in your instruction, and how you will recruit students. In this 
chapter, you will learn how to 

• analyze performance problems, job tasks, and literacy-related job skills; 

• develop customized curriculum, select off-the-shelf materials, and 
develop job ^ds; and 

• plan for student recruitment. 

The first step involved in developing a workplace literacy program is to find 
out where the performance problems exist. This is done by performing a job 
task analysis. The puipose of a job task analysis is to look at the overall picture, 
examining several jobs, and determine where performance problems are 
occurring. In some situations, this may already have been done. For example, 
problems in technical training classes or quality assurance groups may have led 
to a request for additional education. 

After you find out which jobs have performance problems, you can then 
determine which literacy-related skills are involved in those jobs, and which 
specific skills are causing difficulty to workers. This literacy task analysis then 
serves as a guide for helping you design your own customized curriculum. You 
will also have to decide the best mix of designing your own curriculum and 
using off-the-shelf materials. 

How well you recmit learners can have a considerable influence on the 
success of your program. There are several factors that can improve the quality 
of your recruitment. These are providing incentives, using other employees to 
recruit, maintaining visibility, and including employees in the planning process 
so the program meets their needs and interests. 

Conducting a Task Analysis 



To address the needs of tht company and employees; 

• analyze jobs to determine the particular tasks that they require; 

• locate job or task areas that need improvement; and 

• determine if literacy-related job skills need to be improved. 



To make sure your program ties in with the needs of the workplace, you 
will need to become very familiar with the jobs of your target group and 
determine where instruction needs to be focused. When you analyzed the needs 
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of the company (see Chapter Two) you probably found several areas where 
improvement was needed. Did you decide that education can help the company 
and employees meet those particular needs? If so, you now have to decide what 
type of instruction needs to be offered, how it will be done, and to whom it will 
be available. 

To do this, there are two processes that will enable you to examine the areas 
of need and analyze how your program can help address them. One is to 
conduct a large-scale job task analysis of a whole company or department. The 
other is to concentrate on a few jobs where performance problems are known to 
exist. You may not be able to go through the first of Aese processes due to 
limitations of resources or company regulations. However, Iwth are valuable 
tools for helping you build the foundation of an effective workplace literacy 
program. 

In a full job task analysis, you will need to become familiar with all the jobs 
of your target group, whether it is the entire company or a small department. 
This involves examining a group of jobs, and determining and mapping the 
major tasks for each job. For example, if you performed a job task analysis at 
an insurance claims department, you would examine each job in the department, 
determine the major tasks required to perform the job, and map and organize 
those tasks in a chart. You will rarely be asked to look at the whole company 
and find all the performance problems that exist within it. Most often, 
companies will report problems in particular areas, and you will focus on 
finding the specific performance problems in those areas. If you are unable to 
perform a full job task analysis, you will, in any case, want to conduct an 
informal assessment of worker needs by focusing on a few key job tasks. This 
will allow you to move on to the second process of analysis. 

This second process involves looking at specific tasks for each job and 
determining what literacy skills are required to perform each task successfully. 
This is called a literacy task analysis. You would first examine information 
gathered from your job task analysis (whether formal or informal) and 
determine which tasks require literacy-related skills. Then you determine what 
types of literacy skills are required. For example, some literacy-related skills 
required by an insurance claim clerk are asking questions and taking notes from 
conversation, skimming policy manuals and locating information, reading 
manuals for new information, and summarizing information back to the client. 
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ANALYZING PERFORMANCE PROBLEMS 



• Find out the following 

Which tasks are critical to the job? 

What skills need to be enhanced? 

Are there skills associated with promotion? 

Do jobs requite retraining? 

• Gather information through interviews with employees, supervisors 
and managers, observation of workers on the job, and examination of 
printed materials used by workers. 

• Organize your information into a chart. 



When performing a job task analysis, you should examine four aspects of 
the workplace. First, you will want to know which tasks are critical to the job. 
For example, if a worker’s performance in a particular task leads to unsafe 
conditions, then it is critical that he perform this task correctly. Second, find out 
if a number of employees make frequent mistakes in a particular task. If so, 
determine which skills need to be enhanced. Third, find out if there are skills 
that are associated with promotion. Employers often report that they have 
reliable workers who demonstrate desirable traits such as a positive attitude, 
punctuality, and dependability, but who lack the basic skills required for higher 
level jobs. Fourth, determine whether or not the job requires retraining. Many 
workers are returning to the classroom to learn highly technical processes or 
how to operate sophisticated equipment, and they need to learn new academic 
and basic skills to support this. 

Job task analysis includes interviewing various people, observing 
employees on the job, and gathering job-related print materids. By interviewing 
employees, managers, supervisors, human resource personnel, and trainers, 
you can gain a wide range of information about the company and its 
performance problems. Employees can provide information on specific 
performance problems such as what types of discrepancies occur between what 
is policy and what is practiced. For example, if an instruction manual has an 
incorrect or outdated procedure, employees can identify that procedure and tell 
you how they perform it correctly. Supervisors and managers can describe 
performance problems that are occurring and what types of work errors they 
encounter in their own departments. Human resources personnel can provide 
information on the educational requirements for particular jobs and whether 
current employees meet those requirements. Trainers can tell you what types of 
training programs are already in place so you can avoid duplicating efforts. 
Information from all these sources will help you better define the focus of your 
curriculum and build a more credible program. 

The information that you gather in interviews with employees, managers, 
and supervisors will help you choose which jobs to analyze. When selecting 
jobs and job tasks on which to focus the literacy task analysis, look for areas of 
agreement among the various groups. For example, if four of five managers 
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report that entry-level quality control technicians have difficulty reading manuals 
and gauges, ciculating decimals, and recording the average, range, and total 
for Ae product weights, then the quality control technician would be an 
appropriate job to analyze. 

Observing employees at work is also a valuable way of obtaining 
information. After getting permission from the appropriate persons, you may be 
able to shadow employees during a shift and observe them doing their jobs. 
Plan to ask questions and get a go^ understanding of what is done in a certain 
area and how and why it is done. It is a good idea to document your 
observations, if possible, using a video camera, still camera, or cassette 
recorder. Or you might bring along a note taker to help you in recording your 
observations. First make sure that you have the permission of both the 
employee and employer before doing any of these. In many workplaces, 
photography will not be possible for reasons of security or competition. 

A job task analysis is often done in several steps. First, you gather all the 
information you have obtained from interviews, notes, and observations. Then 
you organize the information into a chart format to make it easier to read. The 
following example shows both of these steps. 

EXAMPLE: 

Job: Bank Teller 

Task; Handling customer requests for bank account information 

Step 1 ■ Gathering Information 

The notes you take during an interview with a bank teller might look 
something like this. 

“Talks with customers — says hello, asks questions about their 
request. 

She figures out what information is needed. 

Finds out if they are requesting information or if they have a 
problem. 

Takes notes about what the customer is requesting. 

Looks up information needed from computer files or brochures. 

Delivers information to the customer or helps him solve the 
problem.” 

Step 2 - Organizing the Analysis 

To help organize your task analysis, draw two columns on a sheet 
of paper. In the left column, write a brief description of the tasks. In 
the right column, list the sub tasks of each main task. 
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TASKS SUBTASKS 

Talk with customer. 

Ask the customer how he can be assisted. 
Recognize important information. 



Write down key words, phrases, and 
numbers. 

Ask the customer questions about what he 
needs. 

Determine if he needs information or help 
with solving a problem. 

If information is needed, go to the source 
of information such as the computer files 
or bank brochures. 

Gather the information needed. 

Present information in oral or written form. 



Help the customer solve the problem. 

Suggest follow-up information if 
necessary. 

After conducting the job task analysis, review the information gathered and 
determine which job tasbj are most critical and pressing. To do this, you may 
need to ask questions of experienced workers or supervisors. Ask questions 
like the following: 

• Is the current training addressing the needs of the workforce? 

• Are there changes in job tasks or responsibilities that require more skills? 

• Are there enough workers experiencing problems to justify setting up a 
class? 

Find out where the discrepancies, confusion, or problems exist, and which 
ones can be helped by literacy or basic skills instruction. 



1. Talk with customer. l.l 

1.2 

2. Take notes on 2.1 

conversation. 

2.2 

3. Determine what the 3.1 

customer needs. 

3.2 

3.3 

3.4 

4. Present information to the 4.1 
customer. 

4.2 

4.3 
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Analyzing Literacy-Related Job Skills 



After determining which job areas have performance problems: 

• decide which tasks are most critical and pressing; 

• find out what the specific problems are; and then 

• decide if these problems are caused by a need for improved literacy 
skills. 



By performing a literacy task analysis, you will be able to determine what 
types of literacy skills are needed to perform a job and then design your 
curriculum based on that information. 

A literacy task analysis is a method used to analyze the aspects of job tasks 
that require reading, writing, computation, and problem solving (Drew & 
Mikulecky, 1988; Mikulecky, 1985). It is a detailed account of workers’ 
thought processes when they use print materials on the job. The purpose of a 
literacy task analysis is to identify which performance skills need improvement. 
For example: 

• Are there concerns about safety hazards? 

• Are employees making production errors? 

• Are large amounts of material being wasted due to improper handling? 

By finding out “where the pain” is and what skills are needed to improve the 
situation, you can better identify what types of instruction are needed. You must 
also determine how many workers are experiencing problems. If only one out 
of fifti' workers is having problems, large-scale instruction will not be a cost- 
effective solution. 

When you observe workers carrying out a job task, it is important that you 
obtain a though understanding of how the task is done. For example, if 
workers use information from a form, find out exactly which parts of the form 
are relevant and how the information is used. Keep asking questions to clarify 
your understanding and to prevent the workers from glossing over steps 
because they are very familiar to them. Ask experienced workers or supervisors 
where they see problems occurring and how frequently, and find out as 
precisely as you can what skills are involved in those steps of the task. 
Remember that you will be building curriculum around the skills that workers 
need to learn, using the job materials now being demonstrated to you. 

To help organize your literacy task analysis, use a chart to list a brief 
description of a task and the literacy skills required to perform that task. An 
employee may use several literacy-related skills to complete a task. For 
instance, using the bank teller example of handling customer requests for 
account information, one task related to that job is being able to determine what 
the customer needs. Following is part of a literacy task analysis related to the 
example described previously. 
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EXAMPLE: 

Job: Bank Teller 

Task: Determine what the customer needs. 



Subtasks 

3.1 Ask the customer 

questions about what 
he needs. 

3.2 Determine if he needs 

information or help 
with solving a 
problem. 

3.3 If information is 

needed, go to the 
source of information 
such as the computer 
files or bank 
brochures. 

3.4 Gather the information 

needed. 



Literacy-Related Skills 

3.1.1 Formulate questions about 
the request. 



3.2. 1 Analyze verbal information 
from customer’s responses to 
the questions. 



3.3.1 Scan information resources 
such as brochures or 
computer files. 



3.4.1 Analyze verbal and written 
information. 

3.4.2 Summarize information. 

3.4.3 Take notes. 



Problems may occur with certain subtasks because workers lack the skills 
necessary to perform them. If, for example, some bank tellers are giving 
customers incomplete information, this may be due to their inability to scan 
brochures quickly and efficiently. Knowing this can help focus instruction on 
the techniques of skimming and scanning. 



Building Curriculum From the Literacy Task Analysis 



• Select tasks that cause most difficulty and build curriculum around 
those tasks. 

• Collect samples of print materials used by workers and develop 
curriculum activities from them 



During interviews and observations, you should document the tasks 
involved in doing a particular job. Be sure to collect samples of any literacy 
materials and discuss their importance and frequency of use. You can then use 
the most important materials to help you develop your custom-designed 
curriculum. 
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There are two types of job-related materials: task oriented and general. Both 
include many things that the worker, or you, may not think of initially as 
reading. Examples of task-oriented materials are a work order that tells a 
machine operator which drill to use or a maintenance manual that tells a 
mechanic how many quarts of oil to put in an engine. On the other hand, 
genera/ job-related materials include items that make workers more confident, 
knowledgeable, and valuable to a company, but these items are not needed to 
cany out the job. Examples of general materials include policy manuals, 
insurance information, and retirement fund information. 

After you select the skills that cause the most difficulty, you then design 
curriculum to help workers improve those skills. Develop lessons and activities 
using the materials gathered from the job site that relate to the targeted skills. As 
you identify these sample materials, make copies of them so you can use them 
to design your own curriculum. 

EXAMPLE: The following is one step from the task analysis of the 
bank teller and some possible lessons that could be used to improve 
performance of that task. 

Literacy Skill Applied Possible Lessons 

Summarizes information and To simulate the work enviromnent, use 
takes notes. role play situations. Tellers can practice 

summarizing information and taking 
notes using the bank’s manuals, 
brochures, and computer system. 

Practice note-taking techniques based 
on a customer request using the bank’s 
computer system, manuals, and 
brochures. 



In some cases, you may find that there are few problems in current jobs, but 
problems are anticipated in the near future when jobs will change as new 
technology is implemented or technical training classes are required. In such 
cases, the task analysis should focus on the skills required by these future 
demands. You may need to coordinate with management or supervisors to have 
a demonstration of how a certain job would be accomplished in the future. 



DEVELOPING Curriculum 

When developing curriculum, it is important to 

• use the task analysis to create custom-designed curriculum that links job 
skills and learner interests. 



• select off-the-shelf materials to supplement your custom-designed 
curriculum, 

• determine an appropriate mix of custom-designed materials and off-the- 
shelf materials, and 
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• use job aids to reinforce the instruction. 

Effective workplace literacy curriculum links together current basic skill 
levels, job requirements, and learner interests. When making decisions about 
curriculum, you should consider what mix of custom-designed materials and 
off-the-shelf materials works best. 

DESIGNING INSTRUCTIONAL MATERIALS 



• Design curriculum and instmctional materials-using the literacy task 
analysis and employee interests. 

• Integrate instmction for job tasks with the literacy skills needed to 
perform those tasks. 

• Maintain a balance between job-centered and learner-centered 
instruction. 

• Make sure that you teach correct ideas and do not conflict with other 
training already in place. 



Many literacy program administrators recognize the importance of using 
custom-designed materials. In a survey of 121 workplace literacy program 
reports from the early 1990s, 52 contained information about curriculum and 50 
of those reported some use of custom-designed materials (Mikulecky, Lloyd, 
Horwitz, Masker, & Siemantel, 1995). 

Custom-designed curriculum consists of instructional materials that you 
develop using the literacy job task analysis and additional materials relevant to 
employees’ home skills or personal interests. This usually involves integrating 
the instruction for job tasks with the instruction in literacy skills necessary to 
perform those tasks. Your curriculum is based on the information you have 
obtained from interviews, observations, job materials you have gathered, and 
the hteracy task analysis. 

It is important to maintain a balance between job-centered instruction and 
learner-centered instruction. By using both types of materials, you will capture 
the employees’ interests and encourage them to practice new skills both on the 
job and at home. Learners will also see the relevance of the instruction to both 
their work and personal lives. 

EXAMPLE: At a medium-size machining shop with mainly female 
employees, workers were having trouble following directions in a 
new machine maintenance manual. The literacy instructor used the 
manual as a basis for designing lessons. She copied pages of the 
manual and had the class read and interpret these in a variety of 
situations. Sometimes they worked in pairs, sharing their 
understanding of the instructions. At other times, they worked 
individually, writing their own paraphrases of the manual, which 
they then shared with the class. After the instructor found out that 
several of the employees were interested in cooking, she 
incorporated recipes into the instruction. Since recipes also proceed 
step by step, they helped teach the same skill — following step-by- 
step directions. For each exercise, the instructor modeled how to 
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read the directions, and pointed out similarities and differences 
between the strategies employed in understanding them. 

The type of instruction you present must be valid. Carefully analyze the 
suitability of the materials you create. Ask members of your planning group, 
trainers, supervisors, and a few of your students to review the materials. It is 
important to make sure tliat you teach the correct ideas and do not conflict with 
other training already in place. 

Learners can also help you develop and design the curriculum. By getting 
their input throughout the planning process, you will be able to produce a more 
credible and valued curriculum. They can provide direction in helping you 
define learner interests and how to best incorporate those interests into the 
curriculum. They can also point out discrepancies that occur between policy 
handbooks and their job practices. 

You may find that the number of skills to be taught is more than your 
program can handle in one course. Or you may find yourself in the position of 
not having sufficient time and resources to design all the curriculum you need. 
This is when it is important to consider how off-the-shelf materials or job aids 
can help you further your program. 

SELECTING OFF-THE-SHELF MATERIALS 



• Supplement your customized curriculum with suitable commercial 
materials. 

• Choose materials based on their compatibility with your literacy task 
analysis, your custom-designed curriculum, and the needs of the 
workforce. 

• Supplementary materials include adult educational materials, 
vocational texts, and industry pubUcations. 



Designing your own instructional materials takes time, energy, and 
knowledge. Successful workplace literacy programs contain a good deal of 
customized curriculum, but be careful not to reinvent the wheel. Publishers 
have realized the extensive market in the fields of adult education and vocational 
training. There are many excellent materials that you can use in your courses to 
supplement the curriculum that you design yourself. In addition, many 
programs have entered copies of teaching materials into the ERIC (Educational 
Resources Information Center) database. When you have developed your own 
program, you can assist other workplace educators by sending reports and 
niaterials to the ERIC system. There are several different clearinghouses for 
different aspects of education, and the appropriate one for workplace materials 
is the Clearinghouse on Adult, Career, and Vocational Education. You can 
contact this clearinghouse by telephoning 1-800-848-4815. 

Supplementary instructional materials may be adult education materials, 
vocational texts, or training publications. Choose materials based on their 
compatibility with the literacy task analysis, the custom-designed curriculum, 
and the needs of the workforce. An adequate blend of custom-designed and 
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published materials will provide your students with a job-specific and 
comprehensive course. 

EXAMPLE: In a literacy program at a large insurance company, 
several sources of published materials were used. The primary focus 
of the course was to improve report writing skills. The instructor 
developed customized curriculum on how to write reports properly 
and used the company’s report writing guidelines. She also used 
several types of off-the-shelf materials to help improve grammar and 
writing slalls, such as a book that gave instructions on how to write 
concise paragraphs, an English grammar book, and a dictionary. 

There are many places you can find published materials to use in your 
program. Pc’* example, your planning group may be able to help you find some 
good materials related directly to the industry'. You should also talk with 
professional organizations related to the workplace, and publishers of trade 
journals and magazines. Libraries, vocational or technical schools, and 
bookstores are other good sources. You may also check within your state for 
government agencies, related occupation fields, or private industry groups that 
can offer guidance. 

Curriculum designers, training directors, and publishers of textbooks and 
educational materials are interested in things that work well. Once you have 
located and used various materials, you may find it beneficial to communicate 
with the author or publisher. Let them know how you used it and how it might 
be further developed. Discuss any suggestions for revisions. You will be 
enhancing the field of workplace education as well as guiding the development 
of future materials. 

EVALUATING THE SUITABILITY OF OFF-THE-SHELF MATERIALS 

You may use published materials as part of the core curriculum, to reinforce 
instruction, to serve as a resource guide, or to provide enrichment. To 
determine the suitability of published materials, consider the following things in 
relation to your own methods and goals: 

• intended use of the materials, 

• reading level, 

• method of instruction, 

• amount of coverage of the subject matter, and 

• price of tlie materials. 

Some strands of instruction will rely more heavily on off-the-shelf 
materials, while other strands will lean toward a more customized approach. 
The needs of your target group and the goals and objectives of your program 
will help guide decisions about curriculum development. 

DEVELOPING Job Aids to Further Your Instruction 

• Job aids restructure information to make it more easily understood, so that 
job performance can improve. 

• They simplify job tasks to provide ongoing support after instruction is 
finished. 
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• Learners can help develop job aids as part of practicing new skills. 

A good solution to consider for simplifying job tasks is to use a job aid. Job 
mds, like custom-designed curricula, are based on the literacy task analysis. A 
job aid simply restructures information so it can be more easily understood and 
more accessible to the workers (Mikulecky, 1990). Job aids are designed to 
provide information, reinforce instruction, or provide ongoing support for 
workers after instmction has been completed. 

Although the format may vary, all job aids are designed to improve job 
performance. A job aid may take the format of a flow chart, a checklist, or a list 
of instructions. It may include prompts or clues, or indicate what type of 
information is needed to correct a procedure. 

EXAMPLE: At a large manufacturing compmy, employees were 
being trained to work in self-directed work teams but they were 
having trouble communicating with each other. After providing 
instruction in communication skills, the literacy instructor asked the 
employees to help her make a small job aid listing various 
communication strategies. This not only reinforced the skills and 
provided practice, but also served as a reminder of how to handle 
communication problems after the class had finished. 

Job aids can also be a productive part of a workplace literacy curriculum. 
You can reinforce classroom instruction by utilizing them as part of practice 
time. If workers are expected to improve their basic skills, they must be allowed 
sufficient time to practice these slalls. Using jobs aids provides much needed 
practice time. Getting learners to help you develop the job aid also provides 
expanded practice time. Learners often practice the skill as they help write the 
job aid. They also provide valuable guidance for deciding on the format — 
whether it is a laminated card, a form, or a wallet-size checklist. 

You may also use job aids to teach employees how to do a job. When a new 
skill is introduced, use a job aid to simplify the skill so the worker can 
experience success in the classroom and on the job. They can also be used 
when there is not sufficient time or funds to teach all the employees. 

DEVELOPING A RECRUITMENT PLAN 



• Strategies for building an effective recruitment plan include providing 
incentives for learners, askinjg supervisors and your advisory 
committee for help, maintaining visibility within the company, and 
making sure you are meeting learner interests and needs. 



Effective recmiting is essential to a successful literacy program, especially 
during the first year. More than likely, you will be competing with other 
training and job activities to recmit students for your program. For this reason, 
offer the same class at different times during the year so employees will have 
several opportunities to attend. Heavy work schedules, vacations, or illness 
may prevent someone from taking a course the first time it is offered. 
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Recruiting students for your program requires that you reach all employees 
in your target group. Encourage members of your planning group and odier key 
workplace figures to work with you to recruit students. On-site personnel are 
more familiar to the workforce than you are, and their involvement will lend 
credibility to your program. Several recruitment methods have proven 
successful in workplace literacy programs. These are providing incentives, 
using other employees to recruit, maintaining visibility, and meeting the needs 
and interests of the employees. 

PROVIDING INCENTIVES 

Your job would be easy if all employees signed up for literacy programs 
because they recognized the need to upgrade their skills. The fact is, incentives 
play an important role in enticing people to take part in special training. 
Programs with built-in incentives prove to have better participation than those 
that do not. 

The employer may decide to offer monetary incentives, such as pay for 
attending classes or a bonus paid at the completion of a course. However, 
incentives may take other forms: 

• attendance fulfills training requirements; 

• participants receive special recognition from supervisors; 

• students receive certificates in employment folders; or 

• course completion offers an increased opportunity for upward transition. 

Other incentives that have been successfully employed by workplace literacy 
programs tend to focus on external awards such as free meals, lottery-type 
drawings for the people in classes with prizes that include cash, meals, goods, 
services, and donated gifts. Amenities and support may be provided such as 
child care at the site or refreshments during class. 

While none of these incentives are costly, they send a message of support 
from employee representatives, union, and management. In addition to these 
short-term incentives, there are long-term incentives for employees. These 
include opportunities for them to 

• gain enough skills to maintain their current positions, 

• gain sufficient skills to earn promotions, 

• gain sufficient skills to allow them to seek a new job, and 

• see the value of continuing literacy for themselves and for their children. 

Employees are generally aware of the potential benefits, but they may need 
help seeing how these intrinsic benefits apply to them. 

EXAMPLE: The administrators at an auto manufacturing company 
offered a variety of incentives to program participanU Employees 
attending a technical training strand in statistic^ process control 
received overtime pay for attendance. In the case of incompatible 
class and work schedules, some employees were relieved from their 
work area to attend class. Employees were not required to work 
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mandatory overtime on the day they had class. All participants in the 
program received credit toward mandatory training requirements, 
completion ceitificates in their personnel folders, and recognition by 
the company. 

USING SUPERVISORS TO RECRUIT 

Supervisors of your target group can be helpful in recruiting. Remember 
that supervisors are the individuals who witness the basic skills errors being 
made on the job. They are often the ones who have to correct the errors of their 
workers. The relationship you have with supervisors is vitally important. They 
may be able to recommend attendance in a course to certain employees who are 
having difficulty with particular job tasks. Supervisors can recruit employees 
who receive low performance ratings to help them raise their level of 
performance. They often decide issues regarding shift coverage and overtime 
pay, so it is important to get their buy-in for the program. Their support and 
input means you will present a program that addresses the real problems and 
makes the time spent in class valuable. 

Be aware of the rights and attitudes of employees concerning recruitment. In 
some places, recruitment by supervisors may be welcomed by the workforce. 
However, on other work sites, recruitment by supervisors may be viewed by 
the woilvrorce as giving special treatment to certain individuals or as violating 
employee confidentiality about literacy difficulties. Discuss recruitment 
strategies with managers, union officials, or other decision-makers in the 
workplace. 

BEING VISIBLE 

How you communicate within the workplace depends on the atmosphere of 
the company. In many companies, going to supervisors and employees is very 
helpful. You meet the people and they are able to get to know you. You can talk 
with them about actutd problems and education needs that they see. The more 
you are able to act as an ombudsman and encourager with union representative', 
and supervisors, the more they are likely to share problems, ideas, facilities, 
and opportunities with you. 

However, there are two common problems associated with open recruiting, 
(a) In some situations, people do not want others to know that they are 
attending classes, (b) Going in to a hard hat area entails physical danger to 
yourself and the people working there and may not be possible for insurance 
reasons. In both of these situations, it is preferable to have advocates on the 
floor. 

If confidentiality and safety are not major issues, it will be useful for you to 
become visible to employees. Workers notice even slight changes in the work 
atmosphere. Beginning with your first visit to the work site, engage employees 
in conversation. Show interest in their jobs and introduce yourself to workers 
who are group leaders or shift supervisors. 

Recognize and strive to recruit the leaders within your target group. These 
are employees looked up to by other workers, and if they sign up for a class, 
others may follow. Stress the “buddy system” in your recruitment strategy. 
Recruit co-workers who enjoy each other’s company. 
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Make yourself visible and available to workers often. Schedule information 
sessions before a work shift. Display program information in employee lounges 
or cafeteria settings. Listed below are several other types of communication 
strategies. 

Written or Visual Strategies 

• Place announcements in the company newspaper. It is very helpful if the 
article or announcement can be written by people who have taken the 
instraction being offered. 

• Place fliers with pay checks. 

• Have printed explanations of the company’s training requirements. 

• Use in-house commercial videos, highlighting both the program and the 
coordinator. 

Personal Strategies 

• Have a person in the break room or other convenient location at regularly 
scheduled times to answer questions and talk to workers. 

• Communicate directly with supervisors, so that they can share 
information with their employees. Often supervisors do not know what is 
available. 

• Make sure a person is available regularly to answer questions and to 
counsel employees about classes. 

• Arrange for regular formal and informal “plugs” from union or employee 
representatives. 

EXAMPLE: At a literacy program in a correctional institution, getting 
to know employees was not always easy because of the nature of 
their work. To recruit people for each course, the instructor attended 
roll calls, addressed correctional officers, spoke to department heads 
at their meetings, partnered with the training officer and visited 
various work areas, posted flyers and registration forms in break 
areas, and displayed information in the payroll office. 

MEETING THE NEEDS AND INTERESTS OF LEARNERS 

Including learners in the planning of instruction can also serve as an 
incentive. If they feel that the instruction is relevant and their needs are being 
met, they will be more supportive of your program. It is important that you 
know what their priorities are before making any decisions about the 
curriculum. 

If the company is targeting critical needs for all of its employees, there may 
be company-wide testing. This testing should be followed with personal 
counseling to help employees select what type of education they need. 
Employees often need help in knowing what their test results mean and in what 
order courses should be t^en. 

People need to see “what’s in it for me” before they commit their time and 
energy. For example, a stock room clerk may need to be told that his job is 
changing. Now he, too, must be able to answer customers’ questions since he 
represents the company. In other cases, employees need to see workplace 
literacy as the first step to other courses and future job choices. 
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EXAMPLE: At a manufacturing plant, an interest/needs survey was 
given to employees to spark their interest in a family literacy 
program. An area in the employee lounge was used to display 
materials while a speaker explained the program. In addition, a 
workshop on family stress was conducted in order to find 
employees who might be interested in the program. 

A recruitment plan should be formulated early in the planning stages. A 
workplace literacy program without student support and attendance cannot be a 
success. Be sure to ask members of your advisory team to help you plan your 
recmitment strategies and where you can get support for special advertising 
pieces such as a video or brochure. 

Chapter Summary 

To make sure your program matches the needs of the workplace, you will 
need to analyze the job tasks in the company and focus on the areas where there 
are perform^ce problems. After determining where the performance problems 
exist, you will need to find out if those performance problems are due to a need 
for improved literacy skills. By conducting a literacy task analysis, you can 
determine what literacy skills are needed to perform a job and find out if 
employees need to improve certain skills. After collecting information through 
interviews and observations, you select the skills that cause the most difficulty 
for workers. Use job-related materials to custom design these parts of your 
curriculum. Using off-the-shelf materials can save time and resources, so it is 
important to determine the best mixture of the two types of materials. 
Developing job aids for employees may also be a good solution for simplifying 
job tasks and reinforcing instruction. 

Effective workplace literacy curricula link together current basic skills 
levels, job requirements, and employee interests. It is important to maintain a 
balance between job-centered and learner-centered instruction. By integrating 
the iiistruction, you will capture the employees’ interests and encourage them to 
practice new skills outside the workplace. 

It is also important to develop a recruitment plan to encourage employees to 
attend your classes and make your program a success. Issues such as class 
schedules, employee u'.centives, supervisor and management support, as well 
as learner support, all influence what type of recruitment strategies you use. It is 
important to plan your recmitment strategies early so you can encovu-age student 
attendance and support. 

As you develop your curriculum, keep in mind that you will also need to 
evaluate your program with custom-designed measures. Some of the materials 
gathered in the job task analysis will be the tools you use to evaluate the 
program. Be sure you match the evaluation to the curriculum you have, and the 
curriculum to the goals of the program. Chapter Four shows how to develop a 
program evaluation. 
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Chapter Four ^ 



EVALUATION AND THE LINK TO PRODUCTIVITY 

As you are designing the curriculum for your workplace literacy program, 
you should also consider methods of evaluation. The purpose of ev^uating the 
program is to assess how you are doing and whether the needs of various 
stakeholders are being met. Evaluation should reveal the effectiveness of 
instruction both while it is under way and after the program is completed. This 
chapter discusses the two types of evaluation that you should perform and 
describes in detail the methods for carrying out each of them. The first section 
defines these two t 5 rpes of evaluation — ^formative and summative — ^and indicates 
what each involves. Later sections describe the steps of conducting both types 
of evaluation. A more detailed discussion with sample measures is available in 
Mikulecky and Lloyd (1993) and in Mikulecky and Lloyd (1994b). 

Types of Evaluation 

Evaluation can be grouped into two general categories, formative and 
summative. The purpose of formative evaluation is to check the progress of a 
program while it is still under way so that changes can be made. Summative 
evaluation, on the other hand, assesses the total effectiveness of the program. 
Three areas of concern are addressed in summative evaluation: the program’s 
genera? goals, how much learning took place as a direct result of instruction, 
and the impact on job productivity. 



Formative Evaluation 



• Formative evaluation checks the progress of a program while it is still 
under way, so that changes can be made. 

• It addresses four areas of concern to the program goals, resources, 
instructional processes, and impact. 

• Issues include Do all stakeholders agree on program goals? Are 
resources sufficient to achieve goals? 



Formative evaluation is conducted in the beginning and middle stages of the 
program in order to recommend changes needed to keep the program on track. 
Formative evaluation examines four areas of concern. These are goals, 
resources, processes, and impact. 

Program goals should be shared and understood by all stakeholders, 
including such groups as teachers, learners, managers, union officials, and so 
forth. For instance, are learners aware of the stated goals of the program, and 
do they feel that what they are learning will be relevant to their current or future 
job responsijtjilities? Do teachers and managers agree on the program’s goals? 
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Sometimes, misunderstandings between teachers, learners, and management 
may exist as to what the program is supposed to accomplish. These 
disagreements need to be clarified and resolved before the program continues 
beyond a few weeks of instruction. 

In addition, formative evaluation should address the program’s resources. 
Are sufficient resources available to meet the stated program goals, and is the 
program making reasonable progress toward these goals? Resources include 
class time allocated to achieve goals, appropriate learning materials and space 
for instruction, and instructors with the training and expertise needed to 
accomplish goals. 

Third, are learning processes such as classroom activities, actual learner 
practice time, the pace of learning, and so forth, in line with goals and suitable 
to individual learner needs? . For example, a 30-hour class that is 50% 
discussion with no out-of-class reading is unlikely to achieve much literacy 
gain, though it might expand employee aspirations, oral skills, and self- 
confidence. 

Finally, what is the perceived impact on learners and their ability to do their 
jobs? Do groups of stakeholders have good reason to believe that the program is 
having a positive impact on those it is intended to help? For example, do 
learners believe that attending classes will increase their chances of advancement 
within the company, if that is a stated goal? 

If problems exist with any of these four areas in a literacy program, 
formative evaluation will address these problems and recommend changes that 
could improve chances for success. In addition, it will highlight and report on 
areas where the program is successful. 

EXAMPLE: In a bank, a program was started whose goal was to help 
tellers use complex math and reading skills to process paperwork on 
the job. During formative evaluation, interviews revealed that 
teachers were not using job-related materials to teach these skills, 
but that they were using off-the-shelf materials to teach only general 
math skills. The reason they gave for not using a. more customized 
approach was that no preparation time had been allowed before the 
course was supposed to begin. To solve this problem, the evaluation 
report recommended that the course be interrupted so that instructors 
could gather materials and design exercises that related more directly 
to job tasks. This was not done, but time was allotted for such 
development before the class was offered a second time. 

SuMMATivE Evaluation 



• Summative evaluation assesses the total effectiveness of the program. 

• It examines the program from three main perspectives achievement of 
program goals, learner gains, and impact on company productivity. 
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Summative evaluation considers how successful the program has been in 
meeting its stated goals. Although the majority of the effort involved in 
conducting a summative evaluation will be expended at the end of the 
program — or at some major turning point, if the program is ongoing — it is 
based on assessing learners’ literacy abilities both before and after instmction. 
Thus, assessments should be developed while the curriculum is being designed. 
Using these as pre- and post-measures will allow results to be compared 
directly as a means of measuring gains in literacy abilities. 

Many programs have goals that go beyond increased literacy skills to 
achieving changes in literacy practice outside of class, expanding employee 
educational aspirations, and increasing employees’ senses of personal 
effectiveness with literacy. Interviews and questionnaires conducted before and 
after instruction can assess impact in these areas. 

In addition to measuring gains in literacy ability and life style, summative 
evaluation also examines the impact the program had on productivity in the 
workplace. Productivity indicators such as customer complaints, scrap rates, 
use of suggestion boxes, attendance, and ratings of employees’ job 
performance by supervisors, may show improvement among those who 
attended classes. So, here too, you will need to gather base-line data on learners 
before they start class, in order to measure changes produced by the program. 

Finally, summative evaluation assesses how well the program fulfilled its 
stated goals in terms of providing satisfactory service, producing new reusable 
curriculum materials, and fostering continued learning among employees. 
Program records such as enrollment and retention rates, samples of custom- 
designed instructional materials, samples of learners’ work, records of 
employees enrolling in later classes, and any other material connected to the 
program are gathered and studied. All this information is compiled into a report 
that documents how successful the program has been in assisting learners and 
contributing to productivity. 

EXAMPLE: In an automotive plant, workers enrolled voluntarily in a 
program that combined general education (e.g., GED preparation, 
education in hobbies) with job-related skills instruction. After the 
first year, when two groups had completed classes that included pre- 
and post-assessments (e.g., TABE and GED course completions, 
job task scenarios for job-related instruction), a summative 
evaluation was conducted. In addition to analyzing pre- and post-test 
results, evaluators also examined job attendance, use of the 
suggestion box, and safety violations as measures of productivity. 

These last two were included because instruction had addressed 
reading of safety rules and suggestion writing. Also, samples of 
literacy task analyses and instmctional materials were examined as 
evidence that program goals related to curriculum and instruction 
were being addressed. A report was then written that documented 
program successes and recommended changes for future programs. 

Conducting a Formative evaluation 

• Formative evaluation uses interviews with each group of stakeholders and 
classroom observations to leam how the program is progressing. 
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• The results, conclusions, and recommendations will be submitted to 
program plaimers in a written report while the program can still make 
changes. 

• In addition, an external evaluator can provide an unbiased, expert opinion 
of the program. 

INTERVIEWING STAKEHOLDERS 



• Interview representatives from each group of stakeholders (e.g., 
teachers, managers, ’earners, employee representatives, and 
supervisors). 

• Ask questions about goals, resources, processes, and impact. 

• Keep questions open-ended — do not give your own opinion and then 
seek agreement. 

• If a brief response is given, ask for more examples or reasons; 
continue asl^g for examples until the interviewee runs out of 
responses. 





Formative interviews should be conducted with representativ _,um each 
group of stakeholders including teachers, managers, learners, and others 
involved in planning and implementation. While it is not usually feasible to talk 
to all people involved, you should choose representatives who can discuss the 
program from each group’s perspective. For example, you may want to talk to 
the program manager(s), since there may be only one or two people who hold 
this position. But rather than trying to reach all learners, you should choose a 
representative sample, possibly one or two people from each class. 

Once you have selected participants to interview, you will formulate 
questions to ask interviewees. These questions will be based on goals, 
resources, processes, and impact, as discussed earlier. Typical questions might 
be, “What do you think are the most important goals of the program? What is 
your opinion of these goals?” (See Mikulecky & Lloyd, 1994b for more details 
on conducting a formative evaluation.) Such questions would be asked of all 
interviewees and their answers compared to look for potential disagreements. 
For example, if managers think the goal of the program is current job 
improvement, learners think it is getting a new job, and teachers think it is self- 
development, there may be some problems if the course is too brief to 
accomplish all these goals. 

It is important not to ask leading questions, but to leave them open-ended to 
be answered honestly. Offering hints of the kinds of answers you are expecting 
would be too manipulative. It is acceptable to ask for an opinion of a list of 
items, but it would not be fair to suggest answers if an interviewee gives a brief 
response to the initial question. It is dso unacceptable to state your own opinion 
as part of the question. For example, asking someone, “Well, aren’t the 
materials good?” is not acceptable. If someone does give a brief initial response, 
an appropriate prompt would be to ask for more examples or to ask his opinion 
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of specific items, without hinting. As an interviewer, it is essential to remain as 
neutral as possible to avoid influencing the outcome of the evaluation. 

EXAMPLE: In the automotive plant mentioned earlier, the program 
manager, two teachers, and four students (i.e., two from each of 
two class sessions) were asked whether there were sufficient 
resources to run the program effectively. Specifically, they were 
asked about class time, building facilities, and classroom materials. 

Part of a typical interview with one learner follows: 

Interviewer: “What do you think of the books and other materials used in 
this class?” 

Learner. “OK, I guess.” 

Interviewer. “Can you teU me more?” 

Learner. “Some of them are old and I’m not sure if they are out of date or 
not.” 

Interviewer: “So you’re not sure if you’re getting the most up-to-date 
information?” 

Learner. “Yes.” 

Interviewer. “Is there anything else?” 

Learner: “Well, sometimes we have to share workbooks. We can’t take 
them home because there aren’t enough for everyone to have a copy.” 

Interviewer: “So you think it would be helpful to have your own copies of 
books?” 

Learner. “Yes, I could then take notes in my own book.” 

OBSERVING CLASSROOM ACTIVITIES 



• Observe some classes — ^note the time of each activity and describe events 
as thoroughly as possible. 

• Compare observations with interviews and program goals to determine 
matches and mismatches between what is actu^ly occurring and stated 
goals. 



Eesides interviewing stakeholders, you will also want to observe one or 
more >ypical class sessions. As you observe, take notes and describe activities 
as thoroughly as possible. Be sure to record the time of each activity for later 
refererce. 




The results of these observations will be compared with published program 
goals and interview responses of stakeholders. For example, observations 
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might reveal a difference between what teachers say about the way they spend 
time in the classroom as compared with what they actually do. More typically, 
instruction might address some goals while missing others. Reasons for this 
can be too many goals for a brief course, lack of conununication v/ith the 
teacher about priorities, or lack of resources (e.g., planning time, materials) to 
accomplish goals. 



Analyzing Interviews and Observations 



To analyze the information gathered, look for 

• areas of agreement and disagreement among stakeholders, 

• potential mismatches between goals and resources, and 

• inconsistencies between goals and instmction. 



Once you have completed interviews and observations, you will analyze the 
results. Look for areas of agreement and disagreement among the various 
stakeholders, taking into account the classroom activities you observed. Look 
also for discrepancies between program goals and either resources to 
accomplish them or instructional methods. If problems exist, they are usually 
readily apparent. For example, the manager may believe the program is 
assisting employees with job-related skills, when in fact, teachers are using 
only off-the-shelf materials because they have not been given the time necessary 
to conduct a task analysis before the first class was scheduled to begin. 

EXAMPLE: In the automotive plant mentioned previously, a question 
was asked of managers, teachers, and learners regarding the 
availability of resources. The manager thought that resources were 
more than adequate, but teachers wanted easier access to off-the- 
shelf materials for the GED preparation strand. It was sometimes 
taking several weeks to process purchase orders and have 
workbooks delivered, causing delays for learners who wanted to 
prepare for the upcoming test. Classroom observations confirmed 
that these learners were teing forced to share one workbook among 
several participants, and they could not take materials home for 
additional practice. The evaluator discerned that procurement of 
materials was indeed a problem about which managers were 
unaware, and that materials would have to be made available mote 
quickly so that learners would be adequately prepared for the GED 
test. Such a finding in the evaluation provided the political leverage 
teachers needed to expedite their requests. 

In this example, interviews and observations revealed differing perceptions 
among the various stakeholders. 
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Reporting the Results of Formative Evaluation 



• Prepare a detailed progress report, highlighting strengths and 
limitations, and making recommendations. 

• Have the report reviewed and then distribute it to the planning group 
in advance. 

• Meet with the planning group to present the report — ^be prepared to 
answer questions and discuss implementing recommendations. 



After analyzing the information gathered during interviews, you will want to 
report your findings and recommendations to your planning group- Prepare a 
clear and detailed progress report, highlighting strengths and limitations of the 
program. The findings should be discussed relative to program goals and the 
progress made so far toward reaching them. Include a section containing 
conclusions and recommendations to ensure that the program stays on track. 
You might also want to include a small appendix containing samples of work 
completed by learners. As you describe the findings in the report, be certain not 
to refer to specific individuals. It is important to gain the trust of stakeholders 
by maintaining confidentiality. 

Once the report is written, have another committee member or teacher 
review it for errors, misunderstandings, or other problems. Distribute the report 
to the planning group and schedule a meeting to discuss it. By distributing the 
report in advance, you will avoid problems such as revealing potentially 
negative information for the first time in front of the entire group. At the 
meeting, organize a brief presentation of the results, conclusions, and 
recommendations. Be prepared to answer questions and discuss how to 
implement recommended changes. 

EXAMPLE: In a literacy program at a large insurance company, the 
formative report was written by the program manager and reviewed 
by another instructor. It was distributed to the company’s training 
committee, and a meeting was called to discuss it. The evaluator 
gave a brief presentation of the results, highlighting the program’s 
strengths and problems. Strengths included starting die first class on 
schedule and maintaining a high enrollment due to advance 
publicity. One problem was instructors’ difficulty in obtaining 
materials and supplies in a timely manner. Once the purchasing 
manager was made aware of this problem, she arranged with the 
purchasing department to speed up processing of purchase orders 
for the program. 

Working With an External Evaluator 

Besides conducting an internal evaluation, it is advisable to seek the 
professional advice of an outside evaluator. This expert can provide participants 
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with an independent, unbiased look at the program and recommend changes that 
will assist the program in achieving its stated goals. In addition, he or she can 
monitor the internal evaluation or provide confirmation of results. 

Conducting a formative evaluation will enable the program to remain 
focused on its goals. By interviewing a representative sample of stakeholders, 
discrepancies among the opinions of each group can be uncovered and resolved 
promptly. 



Conducting a Summative evaluation 

• Summative evaluation documents the program’s success in achieving its 
goals. 

• Goals may include attaining gains in literacy ability, improving job 
productivity and attitudes, expanding learners’ educational goals, and 
increasing involvement in literacy activities at home. 

• Pre- and post-test results and other information gathered as part of 
summative evaluation will be used to write the Summative Evaluation 
Report. 

EXAMINING ACHIEVEMENT OF PROGRAM GOALS 



The planning group should 

• review and analyze program goals; 

• examine program records, such as enrollment and retention rates, 
samples of custom-designed instructional materials, and samples of 
learners’ work; and 

• discuss how well goals were achieved. 



To determine how well the program achieved its stated goals, each goal 
should be reviewed and analyzed by the program planning group. Items such as 
relevant pre- and post-test results, meeting minutes, class materials, samples of 
classroom work, and other relevant materials should be gathered as evidence 
that goals were achieved. Informal evidence such as quotes from supervisors 
and written comments by course participants can also be included. 



EXAMPLE: In a food manufacturing plant, the planning committee’s 
goals were reviewed by the group as part of summative evaluation. 
Minutes were taken at the meeting, and these were developed into a 
section in the final report that assessed goal achievement, going 
through the list of stated goals item by item. For example, one goal 
was to interview each worker individually to discuss his or her 
educational needs. Throughout the program, records were kept of 
interviews, and a table showing demographic data about the group 
of interviewees was included in the report to demonstrate that this 
goal was achieved satisfactorily. 
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USING ASSESSMENTS AS PRE- AND POST-MEASURES 



• Develop learner assessments as curriculum is being designed. 

• The assessments should be linked to the learning objectives of the class 
and be relevant to both teaching and jobs. 

• To allow results to be compared directly for evaluation purposes, 
administer the same assessments before and after a course. 

• If a standardized test is used, it should be one that uses realistic tasks 
relevant to the job and life skills included in instruction. 



As curriculum is being designed, you should also develop assessments to 
use as pre-measures of learner ability prior to instruction, and as post-measures 
to determine the extent of gains as a result of instruction. The assessments 
should reflect as closely as possible the objectives of the curriculum being 
taught, and be relevant to both teaching and jobs. (See Mikuleclg^ & Lloyd, 
1994a for examples of possible measures.) So that results can be directly 
compared later, you will administer the same assessment for both the pre- and 
post-tests. Feedback on answers should not be given to learners, or the use of 
the same measures as a post-test will be invalidated. When administering 
assessments, it is important to explain to learners that these are being used to 
measure the program’s achievements and not as an evaluation of learners for the 
purpose of grading them. After course completion, the assessments can be 
scored and the results recorded for later analysis. 

EXAMPLE: A course was developed that taught percentages as they 
related to efficiency reports for production, in the food 
manufacturing plant descril^d above. Objectives included learning 
to read tables and to perform various calculations involving 
percentages and machine efficiencies. Using actual job materials, an 
assessment was developed that measured these skills. It was 
administered before and after instruction so that results could be 
compared and analyzed to assess the extent of gains in learner 
ability. Average scores for the whole class were compared and gains 
were noted in each of the skills addressed by the instruction. 

Standardized tests such as the Test of Applied Literacy Skills (TALS) and 
the CASAS Life Skills Assessment can sometimes be used as part of summative 
evaluation. Those measures are most effective when curriculum matches the 
types of life skills materials and tasks used on these measures. Sometimes only 
a single subtest is needed (e.g., document or quantitative). As with the custom- 
designed measures discussed above, standardized tests should be administered 
as pre- and post-assessments so that results can be compared directly. The 
TALS and CASAS are sometimes appropriate for workplace education because 
each uses realistic tasks (e.g., adding up a bank deposit slip, reading a table). In 
addition, these assessments do not assign a grade level to each learner as is 
done with more general skills tests, such as the Test of Adult Basic Education 
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(TABE) (See Chapter One for a more detailed discussion of TALS and 
CASAS.) 



EXAMPLE: In an auto manufacturing plant, the TALS was 
administered to all workers prior to the start of a class and again 
after each person had completed 20 hours of instruction. The 
average scores on each of the three sections (i.e., prose, document, 
and quantitative) were analyzed to look for gains in learner ability. 
The curriculum addressed a wide range of functional literacy tasks 
from home and the workplace. Some improvement in scores for the 
whole grcap demonstrated that literacy skills in these three areas had 
increased. 

USING COMPANY RECORDS 



• Company records are a good method of assessing the program’s 
impact on productivity. 

• Use records that will allow you to gather information on individual 
employees (e.g., customer complaints, attendance, enrollment in later 
courses). 

• Collect data for a time period before a class and after at least 20-30 
hours of instruction. If possible, compare learners with a control 
group who did not participate in the class. 

• If these records show an improvement in the bottom line, they can be 
used to justify continued financial support. 



To assess the impact of the program on productivity, examine company 
records (e.g., attendance, safety, applying for promotions, use of suggestion 
boxes, error or scrap rates, customer complaints, quality assurance group 
participation). Although the results will be analyzed for the group of learners, 
you will need to gather information on the individuals in class to get an accurate 
representation of any gains in productivity. The items you choose will depend 
on what is available in the company and how records are organized. For 
example, in jobs that involve customer service, records might be kept of 
customer complaints against each employee. Perhaps records are kept of the 
time required to complete paperwork, for jobs that require filling out many 
forms. Attendance records are another source of productivity that may show 
improvement as a result of attending classes. 

It may be difficult to find records that are organized by individuals, but keep 
searching. If you are trying to obtain continued funding, company records can 
be an excellent indicator of the program’s success at improving productivity. 
Programs that can demonstrate an improvement in the company’s bottom line 
are more likely to receive continued support. 

To get around the problem of requiring individual records for each learner, 
it might be possible to teach all the members of a particular department or team 
at one time. Team or dep^ment records could then be examined. For example, 
if team training is the major new factor between pre- and post-class records, the 
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group’s gains in productivity can be attributed to the instruction they all 
received. 

As with other measures, you should collect data on program participants for 
a time period before the program and after a significant amount of instruction^ 
time (e.g., at least 20-30 hours). This data can be analyzed for the group of 
learners to look for improvements in productivity. It may also be useful to 
compare people who attended class with a control group of the same size who 
did not, due to scheduling problems or limits on enrollment, as a way to 
discover whether participation in the class improved the productivity of learners 
when compared to the general worker population. 

EXAMPLE: In a program for correctional officers at a women’s 
prison, attendance records were examined for,Ihose in a report 
writing class and for a control group. Periods of two months before 
and after the class were selected to allow a significant number of 
absences to accumulate. Because the pre-class period of attendance 
was in the fall and the post-class period was in winter, attendance 
fell for both groups. However, the attendance of the control group 
fell significantly more than that of the group in class. Thus, the use 
of the control group could demonstrate the positive impact of the 
report writing class on attendance, whereas without it the impact 
would have appeared to be negative. 

DEVELOPING SUPERVISOR RATINGS 



♦ Supervisor or team leader ratings of employees’ job performance are a 
series of anchored rating scales that are custom-designed to suit your 
company. 

♦ The supervisors or team leaders who will be making the ratings can 
assist you in developing the rating scales of job performance (e.g., 
completing paperwork, working in teams). 

♦ Anchored rating scales help to ensure that pre- and post-ratings are 
made consistently to allow valid comparisons. 



An additional measure of job productivity is the development and 
implementation of job performance ratings of employees attending the program. 
These ratings are a series of anchored rating scales that are custom-designed to 
suit your particular workplace. As is the case with company records, the ratings 
will be used before and after instmction to measure the program’s impact on job 
productivity. Supervisors and possibly key employees will assist you in 
developing ratings on various aspects of job performance, including both 
specific, job-related behaviors and communication indicators related to literacy 
and basic skills. For example, job-related behaviors might include reading 
measurements and setting up equipment, following safety rules or completing 
paperwork. Communication indicators might include behaviors such as how 
well employees communicate as members of a team, and how often they offer 
suggestions at team meetings. 
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It is important to involve the supervisors who will be doing the rating in the 
development of the scales. If supervisors feel they have contributed to the rating 
scales, it is more likely that they will take the rating process seriously, so that 
you can obtain valid evidence that instruction has made a difference to job- 
related behaviors. 

Another factor to consider as you design rating scales is how many 
supervisors from different departments will be using the scales to rate 
employees. When the ratings are scored and the results analyzed, they will be 
reported for the entire group in instruction. The larger the number of employees 
rated, the more accurate will be the analysis. Thus, it is not efficient to design 
separate rating scales for small departments or teams. Rather, the ratings should 
cover a broad range of behaviors that encompass several jobs. For example, 
you might develop a rating scale for the specific communication skills used by 
self-directed work teams and then examine how these skills affected company 
production. 

DESIGNING THE RATINGS IS A FOUR-STEP PROCESS. 

Step One: Ask supervisors or team leaders to describe in detail how 
workers use information on the job. Encourage them to think of 
specific individuals. For example, they might say that a worker 

• uses quality monitoring information, 

• completes paperwork accurately, 

• has information and suggestions to contribute at team meetings, and 

• sets up equipment accurately and checks settings. 

From this list, you will identify areas that can be developed into ratings. 

Step Two; For each of the identified areas, ask supervisors or team 
leaders to think of behaviors that characterize top, bottom, and 
average employees: 

• first, describe the behavior of top performers; 

• then, describe the behavior of bottom performers; and 

• last, describe average performers. 

You will then use these behaviors as descriptions for the anchored ratings. 
The scale will range from 1 to 10, with 10 being the highest possible rating. 
The description of top performers would fall at about 8 or higher on the scale; 
the average description would fall at about 5, and the bottom description would 
ra^ at about 2 on the scale. For example, in relation to paperwork, supervisors 
might agree on the following descriptions: 

Top: completes all job-related paperwork accurately (rating 8); 

Bottom: intimidated by job-re.’.ated paperwork, and avoids it or does it poorly 
(rating 2); 

Average; does job-related paperwork with some mistakes (rating 5). 

Step Three: Once the descriptions are completed, work with supervisors to 
develop initial categories and labels for the rating scales. For example. 
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categories might include paperwork, problem solving, safety, 
responsibility, communication, and so forth. 

Step Four: Draft a rating scale and submit it to supervisors for comments and 
revisions. Sometimes during revision, complex categories may split to 
become two separate scales. An example of two categories from supervisor 
ratings appears below. 

EXAMPLE: 



Employee Assessment-Overall Rating 

Please rate each employee on a scale of 1-10 for each aspect below. 



• An average employee would be rated 5. 

• A top employee would be rated 8 or higher. 

• A bottom employee would be rated 2 or lower. 



EMPLOYEE 

RATER 

PROBLEM SOLVING 
Bottom 

calls supervisor on 
minor details or 
continues to work when 
equipment is faulty 



Average 

makes minor 
adjustments, offers 
solutions to problems 
and calls supervisor 
only When necessary 



DATE 



can analyze job situations, 
make suggestions and 
solutions which implement 
change 



Top 



1 2 


3 


4 5 


6 


7 


8 9 10 


MACHINE SETTING 








Bottom 




Average 




Top 




unable to set machines 
correctly 


usually sets machines 
correctly, but doesn’t 
always check settings 


sets machines correctly and 
checks settings thoroughly 


1 2 


3 


4 5 


6 


7 


8 9 10 
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ANALYZING DATA 



• The results of all pre- and post-assessments should be analyzed to 
look for learner gains. 

• Compare average scores for the group of learners both for each item 
in the assessments and for the total scores in sections of similar 
items. 



The results of all pre- and post-assessments (classroom assessments, 
company records, supervisor/team leader ratings, etc.) should be analyzed to 
determine the extent of learner gains. After all tests are scored and the scores 
recorded, compute the average score for the group of learners. Compare the 
average scores for each item in the assessments and also compare the total 
scores in sections of similar items. If your aim in conducting a summative 
evaluation is to obtain continued funding, you may decide to compare scores 
using a statistical test of significance. If this is so, you may need to consult a 
statistics textbook, the manual for a software package, or an expert in statistical 
analysis. The average scores (or the statistical analysis) will be used to describe 
the extent of learner gains in the Summative Evaluation Report. 

EXAMPLE: In a program for bank tellers, a two-week intensive class 
in using tables (e.g., tables of compound interest rates) was 
developed and implemented. The same assessment was administered 
before and immediately after the course. The average scores were 
compared and an improvement was shown in the ability to use tables 
to look up interest rates. This gain in ability for the group was 
discussed in the sununative report. 

WRITING THE SUMMATIVE EVALUATION REPORT 



The Summative Evrluation Report will discuss 

• attainment of program goals, 

• measures of learner achievement (i.e., pre- and post-assessments), and 

• increases in job productivity (i.e., company records and employee 
ratings). 

The report should 

• explain the goals and learner achievements assessed; 

• describe the methods of assessment; 

• explain the results, highlighting the extent of learner gains; and 

• draw conclusions about the success of the program and recommend 
changes for the future. 
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The purpose of the Summative Evaluation Report is to show what the 
pro^am has achieved. This includes attainment of program goals, learner gains 
in literacy ability, and increases in job productivity. 

In the area of attainment of program goals, briefly explain each goal and its 
intent. Describe how the goal was attained, using tables as supporting evidence. 
If the goal was not achieved to the desired level, explain why. For example, in a 
program in which employee participation is voluntary, there may be a stated 
goal that all employees would be interviewed to determine their interests in 
education. A table indicating how many employees were interviewed, grouped 
by department, would be an efficient means of describing how effectively this 
goal was achieved. 

EXAMPLE: In the food manufacturing plant described earlier, a goal 
of interviewing all hourly workers to determine their inteic-sts was 
agreed upon by the union and management. The goal was achieved 
with the exception of a few employees (9 out of 130), including a 
few who refused to be interviewed and several who left the 
company before interviews could be scheduled. As supporting 
evidence, a table was developed showing how many employees 
from each department were interviewed including a breakdown of 
their interests in education. To avoid revealing the identity of any 
individuals, employees who refused to be interviewed were not 
mentioned directly. 

Second, measures of learner achievement should be explained and the 
results presented. For assessments that were developed for specific courses, 
briefly describe the course, its objectives, and the corresponding assessments. 
Give one or two examples of questions that were asked. Explain the results for 
the group of learners, highlighting any gains in ability. Again, use tables to 
present results for the group of learners. 

EXAMPLE: In the food manufacturing plant, a brief course on reading 
piping diagrams was developed and implemented. Scores on the 
pre- and post-assessments showed an improvement in the ability to 
understand these diagrams and use them on the job. The r 

report stated: 

A two-part short course on reading piping diagrams was given for all 
employees at each shift meeting. The goal of the course was to enhance their 
understanding of piping diagrams which they were required to read on the job. 
A brief exercise was given before and after each class session to determine how 
much was learned. The exercise consisted of a series of piping diagrams. 
Employees were asked to mark the diagram to indicate that they understood 
how to read it. When the scores were compared, employees from all three shifts 
showed an average increase of 12-14% in the ability to read the diagrams. 

Next, discuss the program’s impact on job productivity. Explain the types 
of records that were examined and the types of job behaviors that were rated by 
supervisors. For example, perhaps data on customer complaints about 
individual employees was collected. Explain why this type of record was 
chosen and present the results for the group of learners. Finally, give your 
conclusions about the success of the program including recotiunendations for 
the future. 
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Prepare a draft of the report for the planning comnaittee, and make any 
revisions as necessary. You may also be asked to prepare a presentation of the 
report, as you did for the formative report. Be prepared to discuss the findings 
and your conclusions and answer questions for the group. 



Chapter Summary 

Evaluation should measure the success of the program and whether the 
needs of the various stakeholders are being met. A plan for evaluating the 
program including assessment instruments should be developed as the 
curriculum is designed. There are two types of evaluation, formative and 
summative. Formative evaluation checks the progress of the program while it is 
still under way, so that changes can be made. Summative evaluation assesses 
the total effectiveness of the program, including how successful it is in meeting 
stated goals, in assisting learners through instruction, and in improving job 
productivity. It uses assessments as pre- and post-measures of achievement. 
Summative evaluation also looks at company records (e.g., attendance, 
customer complaints) to assess the impact on job productivity. 

To conduct a formative evaluation, representatives from each group of 
stakeholders are interviewed to determine the extent of agreement or 
disagreement in the four areas of goals, resources, processes and impact. In 
addition, several classes are observed and observations are compared to 
interview result^ A draft report is written and submitted to one or two 
individuals for review before being distributed to all stakeholders. Discussion 
of the results and conclusions should help the planning committee to make 
appropriate changes to keep the program on track. 

Summative evaluation is carried out by examining program goals, 
measuring learner achievement through custom-designed assessments and 
standardized tests, and looking at productivity indicators (e.g., supervisor 
ratings and company records). Each of these measures is scored and the average 
scores for the group of learners is calculated. The results are then compiled in 
the Summative Evaluation Report. The report discusses achievement of 
program goals, learner achievement, and job productivity. The report draws 
conclusions about the overall success of the p.:ogram and recommends changes 
for the future. 



lMPLICAm)NS AND CONCLUSION? 



In an increasingly global economy, the demands on workplaces to serve 
customers and compete on a world level are growing all the time. As businesses 
face these challenges, literacy demands also continue to expand. If workplace 
literacy programs are to be effective in meeting these demands, they must serve 
the literacy needs of both employees and businesses. 

Two factors that are key to a program’s success are obtaining the buy-in of 
stakeholders and developing multiple strands. To establish a successful 
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workplace literacy program, it is essential to obtain the buy-in of all 
stakeholders. Goals must be understood and agreed upon by management, 
union, instructors, learners, and anyone else who will be involved with the 
program. 

Besides obtaining the buy-in of stakeholders, the program should work 
towards providing multiple strands of instruction that will address both 
learners’ needs and the demands of jobs. Some strands can be custom-designed 
to solve job-related skill problems. Additional strands can then be developed 
that provide for the long-term needs of learners. Workers could enroll in a 
program strand that addresses low-level skills, then advance to a job-related 
skills course or possibly a GED preparation course, and finally, enroll in 
technical courses at a local community college. Thus, an effective program can 
be developed that will have an impact on the organization’s bottom line and also 
assist learners with personal goals such as obtaining the GED certificate or 
advancing to higher paying jobs. 

Some program strands will use off-the-shelf materials, but others will be 
custom-designed to address learner needs and job-related skills. The materials 
you custom-design will be based on a literacy task analysis of key job areas, so 
that instruction will relate directly to performance difficulties. Literacy task 
analysis involves interviewing and observing workers in the selected jobs, 
breaking down jobs into tasks and subtasks, and analyzing the literacy-related 
skills involved. The literacy task analysis along with Ae materials workers use 
on the job will become the basis for teaching these literacy-related job skills. In 
addition to custom-designing parts of the curriculum, you will use some off- 
the-shelf materials and some instruction that considers learners’ interests (e.g., 
completing insurance claim forms, reading to children, or job advancement). An 
effective workplace literacy curriculum links together current basic skill levels, 
job requirements, and learner interests. 

A key element of program planning is evaluation. Formative evaluation can 
provide valuable information about how things are going and point out areas of 
concern that can be addressed while the program is still in progress. Summative 
evaluation can establish accountability for the program, providing evidence for 
continued funding if necessary. This type of evaluation examines the 
achievement of program goals. It is based on assessments relevant to the 
curriculum that measure learner gains, and includes the achievement of 
instructional objectives and increased job productivity. 

The issue of workplace literacy presents many complex challenges and 
problems. You need to be able to operate flexibly with a variety of stakeholders, 
some of whom may not clearly understand just what a workplace literacy 
program can and cannot do. This handbook offers solutions to these concerns 
and provides practical suggestions for establishing a program that is both 
achievable and likely to produce positive results. You will hopefully find it 
useful as a general guide to building a successful program. Take up the 
challenge and enjoy your successes. 
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Further Resources 



Camevale, A. P., Gainer, L. J., & Meltzer, A. S. (1988). Workplace basics: 
The skills employers want. Alexandria, VA: American Society for Training and 
Development. 

This 35-page booklet describes the nature of workplace basic skill 
demands and outlines the process of establishing a custom-designed 
program. It is available from American Society for Training and 
Development, 1630 Duke Street, Box 1443, Alexandria, VA 22313. 

Drew, R. A., & Mikulecky, L. J. (1988). How to gather and develop job 
specific literacy materials for basic skills instruction. Bloomington, IN: Office 
of Education and Training Resources, School of Education, Indiana University. 
(ERIC Document Reproduction Service No. ED 297 160) 

This 70-page guide describes in detail how to conduct a task analysis, and 
includes examples of task interview forms and how to use workplace print 
materials in teaching. It is available from Vocational Education Services, 
840 State Road 46 Bypass, Indiana University, Bloomington, IN 47405. 

Henard, D., Lloyd, P., & Mikulecky, L. (1992). A guidebook for developing 
workplace literacy programs. Indianapolis, IN: Office of Workforce 
Development. (ERIC Document Reproduction Service No. ED 348 580) 

This 120-page guide leads the reader through the full process of 
establishing a workplace program, from setting up a planning group to 
evaluating die results. It includes illustrative examples and anecdotes from 
the State of Indiana Model workplace program, and is available free from 
the Office of Workforce Development, Indiana Government Center, 
Indianapolis, IN 46204-2277. 

Mikulecky, L., & Lloyd, P. (1993). Evaluating the impact of workplace literacy 
programs (Tech. Rep. No. TR93-02). Philadelphia: University of 
Pennsylvania, National Center on Adult Literacy. (ERIC Document 
Reproduction Service No. ED 348 579) 

This 170 page book describes a new model for evaluating workplace 
programs and includes detailed examples of assessment measures, as well 
as &e background to the development and implementation of the model. It 
is available, in either hard copy or electronically, from the National Center 
on Adult Literacy, University of Pennsylvania, 3910 Chestnut Street, 
Philadelphia, PA 19104-3111. 

Mikulecky, L., & Lloyd, P. (1994a). Handbook of ideas for evaluating 
workplace literacy programs. Bloomin^on, IN: Indiana University School of 
Education. (ERIC Document Reproduction Service No. ED 375 264) 

This 50-page handbook provides a menu of techniques for evaluating 
workplace literacy programs and includes detailed examples of assessment 
measures, how to develop and conduct them, and how to assess the 
results. 

Mikulecky, L., & Lloyd, P. (1994b). Indiana ’s evaluation model for adult basic 
education programs: A handbook. Bloomington, IT^: Indiana University School 
of Education. (ERIC Document Reproduction Service No. ED 375 265) 
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This 350-page handbook provides a series of workshops on formative 
and summative techniques of evaluation. It includes full details of the 
evaluation measures used in the Indiana model, how to conduct them, 
how to assess the results, and how to write formative and summative 
reports. 

Mikulecky, L., et al. (1992). Developing Workplace Literacy Programs. (A set 
of five brochures.) Philadelphia: University of Pennsylvania, National Center 
on Adult Literacy. (ERIC Document Reproduction Service No. ED 348 578) 
This series of five brochures outlines the key issues involved in setting up 
a workplace program: Gaining Management Support, Working with 
Management and Unions, Discussing Training Needs, Recruiting 
Students, and Planning Ahead. 

Philippi, J. W. (1991). Literacy at work: The workbook for program directors. 
New York: Simon & Schuster. 

This 300-page guide describes in detail how to set up a workplace 
program including task analysis and custom-designing. It is available, for 
$200, from Simon & Schuster (1-800-223-2336). 

Taylor, M., & Lewe, G. (1990). Literacy task analysis: A how-to manual for 
workplace trainers. Ottawa, Ontario: Algonquin College, Adult Basic Education 
Department. (ERIC Document Reproduction Service No. ED 337 580) 

This 170-page manual describes in great detail how to carry out a task 
analysis, with samples of forms to use, plus advice on developing training 
from the analysis. It is available from Glenda Lewe, Suite 2301, 530 
Laurier Avenue West, Ottawa, Ontario, KIR 7T1. 

Taylor, M. C., Lewe, G. R., & Draper, J. A. (Eds.) (1991). Basic skills for 
the workplace. Toronto, Ontario: Culture Concepts, Inc. 

This 500-page book contains chapters on the demand for workplace 
programs, on how to assess worker needs, conduct a task analysis and 
design curriculum, on example case studies, and on teaching methods and 
evaluation. It is available from Culture Concepts, Inc, 5 Darlingbrook 
Crescent, Toronto, Ontario, M9A 3H4. 
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Appendix 



This appendix contains a set of overheads that provide a broad outline 
sununary of each chapter. These are intended to serve as teaching aids so that 
the guide can be presented as a series of workshops for staff development. This 
could be done chapter by chapter or section by section, depending on the time 
available and the needs of program staff. 
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Current Practices in Workplace Literacy 
Programs 

What we know about workplace literacy programs 

• There are several different workplace literacy 
problems which call for different sorts of 
solutions. 

• Improvement takes a significant amount of 
learner practice time. 

• Transfer of learning to new applications is 
severely limited. 

• Significant learning loss occurs within a few 
weeks if skills are not practiced. 
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Labor Market Trends 

• A shift in employment from relatively low-skilled 
jobs in manufacturing to higher-skilled jobs in the 
service sector. 

• A trend toward more efficient ‘‘high- 
performance” workplaces. 

• An increase in the use of literacy in workplaces. 

• An increase in temporary and part-time work. 

• A shortfall in the level of skills workers bring to 
the job. 

• An increase in the number of workers with little 
ability to speak and understand English. 
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Labor Market Trends (continued) 

The Shift from Manufacturing to the Service Sector 

• Since 1965, the United States has experienced a 
shift in employment from manufacturing to 
service sector jobs: 

1965 1993 

manufacturing 29.7% 16.3% 

service sector 63.9% 79.0% 

• In the past, it was possible to earn more in 
manufacturing jobs that demanded less skill. 

• Many workers find that they must now have 
higher literacy skills to get jobs that pay less. 

• This can have serious motivational consequences 
for those workers, which must be addressed by 
workplace educators. 
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Labor Market Trends (continued) 

The Growth in High-Performance Workplaces 

• Increasing numbers of businesses are adopting 
high-performance techniques, such as quality 
assurance teams and greater use of technology. 

• High performance workplaces have greater 
demand for higher- skilled workers. 

• Fewer jobs are available but with more 
responsibility. 

• Education should focus on skills that will help 
workers cope with additional responsibilities. 
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Labor Market Trends (continued) 

Increased Use of Literacy in the Workplace 

• Workplaces use a combination of 

- prose literacy, 

- document literacy, and 

- quantitative literacy. 

• Different skills are often used together. 

• Instruction should use realistic simulations to 
provide a link to the way skills are used on the 
job. 
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Labor Market Trends (continued) 

Increase in Temporary and Part-time Work 

• Due to the ever-increasing costs of long-term 
employment commitments (e.g., benefits), 
employers are less likely to invest in permanent, 
full-time employees. 

• Individual workers will have more major job 
changes over their lifetimes. 

• Temporary workers face higher and more varied 
literacy requirements as they move from job to 
job. 

® They will require education Ui flexible skills, 
such as using resources to solve problems and 
answer questions on the job. 
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Worker Populations 

Low Levels of Worker Skills 

• The National Adult Literacy Survey (NALS) 
shows that one half to three fourths of workers in 
certain job categories perform at the lower two 
levels of the survey. 

Categories include: 

- transportation 

- service 

- fanning 

- laborer 

• Job literacy requirements are increasingly at 
Level 3 or above. 

• Workplace instruction should focus on these more 
demanding job tasks and build in practice of the 
necessary skills. 

• Opportunities should be provided for long-term 
education for learners at the lowest levels if 
improvement is expected. 
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Worker Populations (continued) 

Workers With Little English 

• Workers with little English are 

“ over-represented in lower skilled jobs; 

- less likely to be provided with education from 
employers; and 

- unemployed more often and have greater needs 
for education due to turnover. 

• Such workers do not all have the same need for 
instruction in reading and writing, listening and 
speaking, 

• Workers from other cultures may need help in 
making the shift to American workplace 
behaviors, attitudes, and expectations. 

• W^orkers with little English are likely to learn 
more quickly if they are taught in the context of 
the job and outside interests. 





Characteristics of Effective Programs 

Multi-Strand Programs 

• Multi-strand programs should provide for diverse 
populations on a long-term basis. 

• They should also address the needs of low-level 
learners by providing a sequence of learning 
experiences from basic to more advanced 
instruction. 

• A multi- strand program can expand from small 
beginnings to meet the needs of dl workers. 




Characteristics of Effective Programs 
(continued) 

Involvement of Major Stakeholders 

• The active involvement of major stakeholders is 
important to the success of the program. 

• These stakeholders include 

- management, 

- employee organizations, 

- instructors, and 

- learners. 

• A good way to achieve this involvement is to 
form an advisory committee made up of 

— ^epfe^tatives from the key groups in the 
workplace. 

• Involvement in the planning stages increases the 
likelihood that stakeholders will buy in to the 
program. 
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Characteristics of Effective Programs 
(continued) 

Custom-Designed Curriculum 

Custom-designed curriculum: 

• takes learner interests into account; 

• is based on analysis of job tasks and skills; and 

• uses materials from workers’ jobs. 
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Characteristics of Effective Workplace Literacy 
Programs 

• Multi-strand programs are an effective way to 
meet the demands of the diverse populations 
within a workplace. 

• Custom-designed materials integrate basic skills 
instruction with workplace instruction. 

• Forming an advisory committee involves major 
stakeholders in the planning process. 

• Planning for long-term educational goals allows 
workers — and the program — ^to grow. 



95 




Needs Analysis 

• “Where is the pain?” — 

what problems are causing real concern for the 
employees and company. 

• Gather information through interviews and 
surveys, and by observing daily operations in the 
workplace. 

• Determine which areas of company performance 
need improvement and find out the differences 
between where the company and the workers are 
and where they want or need to be. 

• Define which problems are most pressing and 
determine whether they can be solved through 
education. 
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Designing a Workpiace Literacy Program 

• Address the needs of both the employees and the 
company. 

• Create a balance of workplace demands and 
learner interests, 

• Offer multi-strand programs to meet the needs of 
various workplace groups, 

• Involve workers in developing and recruiting for 
your program. 
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Addressing the Needs of Various Groups 



• Multiple program strands are needed to address 
multiple problems. 

• Improvement takes significant learner practice 
time, and varies greatly depending on the learners 
and the material. 

• For example, learners can be 

- high school graduates, 

- high school non-graduates, 

- part-time employees, 

- low-level literates, or 

- English as a second language (ESL) students. 
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Making Use of Non-Training Soiutions 



• Non-training solutions include: 

- providing job aids, 

- redesigning forms, and 

- rewriting instructions. 

• But be careful — employers will not always be 
open to these types of changes. 

• Non-training solutions help increase the 
effectiveness of your program and help 
employees perform their jobs better. 

• Consider the cost of developing and providing 
non-training solutions and the alternatives that are 
available. 
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Resources and Logistics 

• Funding sources include state and federal grants, 
partnerships with libraries and colleges, 
workplace unions and management. 

• Program teachers are a crucial element and may 
need help with using job-related curriculum and 
becoming sensitive to the workplace culture. 
Above all, they must be flexible. 

• Look for ways to share resources with unions, 
companies, and training departments. 

^ • Discuss the scheduling of classes around the 
company’s activities to achieve maximum 
cooperation. 
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Resources and Logistics (continued) 

Finding Funding Sources 

These include: 

• state level grants 

• federal literacy grants 

• unions 

• partnerships with local libraries, colleges, and 
volunteer organizations 

Be aware that... 

• outside funds may nt d 100% employer match, 

• federal funding is usually seed money to start 
programs, and 

• there may be a delay in obtaining funds. 
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Resources and Logistics (continued) 

Staff Development 

• Program teachers are vitally important — a 
program depends on how teachers interact with 
learners. 

• Teachers may need help with using job-related 
curriculum, particularly if they come from ABE 
or school backgrounds. 

• Teachers need to learn as well as teach — sharing 
responsibility with workers who are content 
experts on their jobs. 

• Teachers need to be sensitive to the workplace 
culture and adapt instruction to workers’ 
immediate needs. 

Above all, teachers must be flexible. 
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Resources and Logistics (continued) 

Allocating Your Resources 

Resources include: 

• facilities 

• funds 

• supplies 

• storage space 

• services such as advertising 

• relationships with officials 
Some tips to maximize resources: 

• Establish relationships with other departments 
and unions to share resources 

• Use loaned resources 

• Plan the logistics of your program early — 
schedule and location of classes, storage space, 
location of office 

• Ask learners to help advertise the program 
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Resources and Logistics (continued) 

Planning Program Schedules 

• Class times must be as convenient as possible for 
everyone without interfering with business. 

• Schedule classes before and after shifts. 

• Develop take-home lessons for learners. 

• Use computer-guided instruction. 

• Start courses when the company has a slower 
production schedule. 

Be flexible — work with supervisors, managers, and 
employees to find out what is most convenient 
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Gaining Management and Employee Support 

• Balance education and productivity needs. 

• Involve management and employees in program 
development. 

• Decide together what needs can be addressed first 
with the time, money, and resources available. 

• Develop a shared vision for the future. 
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Maintaining Reiations with Empioyees, 
Management, and Unions 



• Successful outcomes and satisfied clients are 
achieved by reaching jointly agreed upon and 
reasonable goals. 

• Constantly review the program’s progress with 
supervisors, managers, and unions. 

• Keep the lines of communication open, if 
possible through an in-house liaison person. 

• Keep accurate records for your program on 
finance, and learner attendance and progress. 
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Suggestions for Maintaining a Successful 
Program 

• Create a positive, actively involved relationship 
with employees, management, and union 
officials. 

• Understand the need for flexibility: 

- anticipate and plan for problems, and 

- find creative solutions. 

• Meet regularly with joint advisory teams to make 
sure clients are satisfied with the directions and 
outcomes of the program. 
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Conducting a Task Analysis 

To address the needs of the company and employees: 

• analyze jobs to determine the particular tasks they 
require; 

• locate job or task areas that need improvement; 
and 

• determine if literacy-related job skills need to be 
improved. 




Analyzing Performance Problems 

• Find out: 

- which tasks are critical to the job? 

- what skills need to be enhanced? 

- are there skills associated with promotion? 

- do jobs require retiaining? 

• Gather information through 

- interviews with employees, supervisors, and 
managers; 

- observation of workers on the job; and 

- examination of printed materials used by 
workers. 

• Organize your information into a chart. 
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Example from a Job Task Analysis 

Tasks Subtasks 

3. Determine what 3.1 Ask the customer questions 
the customer about what he needs, 

needs 

3.2 Determine if he needs 
information or help with solving a 
problem. 

3.3 If information is needed, go to 
the source of information such as 
the computer files or bank 
brochures. 

3.4 Gather the information needed. 



After determining which job areas have performance 
problems: 

• decide which tasks are most critical and pressing, 

• find out what the specific problems are, and 

• decide if these problems are caused by a need for 
improved literacy skills. 



• A method used to analyze the aspects of job tasks 
that require reading, writing, computation, and 
problem solving. 

•To help determine what types of literacy skills are 
needed to perform a j -. 
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Example from a Job Task Analysis (continued) 

Example: 



Subtask 

3.4 Gather the information 
needed. 



Literacy-Related Skills 

3.4.1 Analyze verbal and 
written information. 

3.4.2 Summarize 
information. 

3.4.3 Take notes. 
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Building Curriculum from the Literacy Task 
Analysis 

• Select tasks that cause most difficulty and build 
cuniculum around those tasks. 

• Collect samples of print materials used by 
workers and develop curriculum activities from 
them. 

Example: ' 

Literacy Skills Applied 

Summarize information and take notes. 

Possible Lesson(s) 

To simulate the work environment, use role play 
situations. 

Tellers can practice summarizing information and taking 
notes using the bank’s manuals, brochures, and 
computer system. 
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Selecting Off-the-Shelf Materials 



• Supplement your customized curriculum with 
suitable commercial materials. 

• Choose materials based on their compatibility 
with 

- your literacy task analysis, 

- your custom-designed curriculum, and 

- the needs of the workforce. 

• Supplementary materials include 

- adult educational materials, 

- vocational texts, and 

- industry publications. 
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Developing Job Aids 

• Job aids restructure information to make it more 
easily understood, so that job performance can 
improve. 

• They simplify job tasks to provide ongoing 
support after instruction is finished. 

• Learners can help develop job aids as part of 
practicing new skills. 

• Job aids include 

- flow charts, 

- checklists, and 

- lists of instructions. 
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Developing a Recruitment Plan 

Strategies for building an effective recruitment plan 
include 

• providing incentives for learners, 

• asking supervisors and your advisory committee 
for help, 

• maintaining visibility within the company, and 

• meeting learner interests and needs. 

Communicate to workers through 

• announcements in company newspaper, 

• fliers in paychecks, 

• talking with employees and supervisors, 

• in-house videos that explain programs, and 

• arranging for “plugs” from employee 
representatives. 
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Evaluation and the Link to Productivity 

Types of Evaluaxion 

Formative evaluation checks the progress of a program 
while it is still under way, so that changes can be made. 

• It addresses four areas of concern to the program: 

goals, resources, instructional processes, and 
impact. 

• Issues include the following: 

- do all stakeholders agree on program goals? 

- are resources sufficient to achieve goals? 

Summative evaluation assesses the total effectiveness of 
the program. 

• It examines the program from three main 
perspectives: 

- achievement of program goals, 

- learner gains, and 

- impact on company productivity. 
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Conducting a Formative Evaluation 

• Formative evaluation uses interviews with each 
group of stakeholders and classroom observations 
to learn how the program is progressing. 

• Results, conclusions, and recommendations will 
be submitted to program planners in a written 
report while the program can still make changes. 

• In addition, an external evaluator can provide an 
unbiased, expert opinion of the program. 
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Conducting a Formative Evaluation (continued) 

Interviewing Participants 

• Interview representatives from each group of 
stakeholders: (e.g., teachers, managers, learners, 
employee representatives, supervisors). 

• Ask questions about goals, resources, processes, 
and impact. 

• Keep questions open-ended — do not give your 
own opinion and then seek agreement. 

• If a brief response is given, ask for more 
examples or reasons. 

• Continue asking for examples until the 
interviewee runs out of responses. 
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Conducting a Formative Evaluation (continued) 

Observing Classroom Activities 

• Observe some classes — note the time of each 
activity and describe events as thoroughly as 
possible. 

• Compare observations with interviews and 
program goals to determine matches and 
mismatches between what is actually occurring 
and stated goals. 

Analyzing Interviews and Observations 

To analyze the information gathered, look for 

• areas of agreement and disagreement among 
stakeholders, 

• potential mismatches between goals and 
resources, and 

• inconsistencies between goals and instruction. 
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Conducting a Formative Evaluation (continued) 

Reporting the Results 

• Prepare a detailed progress report, highlighting 
strengths and limitations, and making 
reconmiendations. 

• Have the report reviewed and then distribute it to 
the planning group in advance. 

• Meet with the planning group to present the 
report — be prepared to answer questions and 
discuss implementing reconunendations. 

Working With an External Evaluator 

• An external evaluator can provide an 
independent, unbiased opinion of the program 
and recommend needed changes. 
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Conducting a Summative Evaluation 



• Summative evaluation documents the program’s 
success in achieving its goals. 

• Goals may include 

- attaining gains in literacy ability, 

- improving job productivity and attitudes, 

- expanding learners’ educational goals, and 

- increasing involvement in literacy activities at 
home. 

• Pre- and post-test results and other information 
gathered as part of summative evaluation will be 
used to write the Summative Evaluation Report. 
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Conducting a Summative Evaluation (continued) 

Examining Achievement of Program Goals 

The planning group should 

• review and analyze program goals; 

• examine program records, such as enrollment and 
retention rates, samples of custom-designed 
instructional materials, and samples of learners’ 
work; and 

• discuss how well goals were achieved. 
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Conducting a Summative Evaluation (continued) 

Using Assessments as Pre- and Post-Measures 

• Develop learner assessments while curriculum is 
being designed. 

• The assessments should be linked to the learning 
objectives of the class and be relevant to both 
teaching and jobs. 

• To allow results to be compared directly for 
evaluation purposes, administer the same 
assessments before and after a course. 

• If a standardized test is used, it should be one that 
uses realistic tasks relevant to the job and life 
skills included in instruction. 
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Conducting a Summative Evaluation (continued) 

Developing Supervisor Ratings 

• Supervisor or team leader ratings of employees’ 
job performance are a series of anchored rating 
scales that are custom-designed to suit your 
company. 

• The supervisors or team leaders who will be 
making the ratings can assist you in developing 
the rating scales of job performance (e.g., 
completing paperwork, working in teams). 

• Anchored rating scales help to ensure that pre- 
and post-ratings are made consistently to allow 
valid comparisons. 
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Conducting a Summative Evaluation (continued) 

Analyzing Data 

• The results of all pre- and post-assessments 
should be analyzed to lookTor learner gains. 

• Compare average scores for the group of learners 
both for each item in the assessments and also for 
the total scores in sections of similar items. 
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Conducting a Summative Evaluation (continued) 

Writing the Summative Evaluation Report 

The Summative Evaluation Report will discuss 

• attainment of program goals, 

• measures of learner achievement (i.e., pre- and 
post-assessments), and 

• increases in job productivity (i.e., company 
records and employee ratings). 

The report should 

• explain the goals and learner achievements 
assessed; 

• describe the methods of assessment; 

• explain the results, highlighting the extent of 
learner gains; and 

• draw conclusions about the success of the 
program and recommend changes for the future. 
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